Nov 04 05:23:59.259081 ip-10-0-133-143 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Nov 04 05:23:59.259095 ip-10-0-133-143 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Nov 04 05:23:59.259105 ip-10-0-133-143 systemd[1]: kubelet.service: Failed with result 'resources'. Nov 04 05:23:59.259453 ip-10-0-133-143 systemd[1]: Failed to start Kubernetes Kubelet. Nov 04 05:24:10.679663 ip-10-0-133-143 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Nov 04 05:24:10.679677 ip-10-0-133-143 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 9a27b69e26d74df4a73876bc959f0dcd -- Nov 04 05:26:27.198970 ip-10-0-133-143 systemd[1]: Starting Kubernetes Kubelet... Nov 04 05:26:27.654866 ip-10-0-133-143 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:26:27.654866 ip-10-0-133-143 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 05:26:27.654866 ip-10-0-133-143 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:26:27.654866 ip-10-0-133-143 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:26:27.660181 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660096 2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:27.660181 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660180 2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:27.660181 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660184 2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660187 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660190 2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660193 2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660196 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660199 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660201 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660204 2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660207 2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660210 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660212 2573 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660215 2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660227 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660230 2573 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660233 2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660236 2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660240 2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660245 2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660248 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660251 2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:27.660272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660254 2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660257 2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660260 2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660263 2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660265 2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660268 2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660271 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660273 2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660277 2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660280 2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660283 2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660286 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660288 2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660293 2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660297 2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660300 2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660302 2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660305 2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660308 2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:27.660767 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660311 2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660313 2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660316 2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660318 2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660321 2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660324 2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660326 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660329 2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660332 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660334 2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660338 2573 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660341 2573 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660343 2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660346 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660349 2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660351 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660354 2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660357 2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660360 2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660362 2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:27.661233 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660365 2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660368 2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660370 2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660373 2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660375 2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660378 2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660380 2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660383 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660385 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660388 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660391 2573 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660394 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660396 2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660399 2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660402 2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660405 2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660407 2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660410 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660413 2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:27.661708 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660416 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660419 2573 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660422 2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660424 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660427 2573 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660429 2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660432 2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660434 2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660437 2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660440 2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660443 2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660446 2573 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660448 2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660451 2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660453 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660456 2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660459 2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660461 2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660464 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660467 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:27.662157 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660470 2573 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660472 2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660475 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660478 2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660482 2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.660484 2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661296 2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661305 2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661310 2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661313 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661316 2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661319 2573 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661322 2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661326 2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661330 2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661333 2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661335 2573 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661338 2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661341 2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:27.662630 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661344 2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661347 2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661350 2573 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661353 2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661356 2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661359 2573 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661362 2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661365 2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661368 2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661371 2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661374 2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661377 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661380 2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661382 2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661385 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661387 2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661392 2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661395 2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661400 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:27.663095 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661402 2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661405 2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661409 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661411 2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661414 2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661417 2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661420 2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661422 2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661426 2573 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661428 2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661431 2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661434 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661437 2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661439 2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661442 2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661445 2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661447 2573 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661450 2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661453 2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661457 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:27.663543 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661459 2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661462 2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661465 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661468 2573 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661471 2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661474 2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661477 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661479 2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661482 2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661485 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661488 2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661490 2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661493 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661496 2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661498 2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661501 2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661504 2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661506 2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661510 2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661512 2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:27.664030 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661515 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661518 2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661521 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661524 2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661527 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661529 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661533 2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661536 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661538 2573 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661541 2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661545 2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661548 2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661550 2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661553 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661556 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661559 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661561 2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661564 2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661566 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:27.664514 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661569 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661572 2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661575 2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661577 2573 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661580 2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661582 2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661585 2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661588 2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661590 2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661593 2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661596 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661599 2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661601 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661604 2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.661606 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662109 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662114 2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662119 2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662123 2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:27.664981 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662126 2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662129 2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662132 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662136 2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662138 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662141 2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662144 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662146 2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662149 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662152 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662154 2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662157 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662160 2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662163 2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662166 2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662168 2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662171 2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662174 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662177 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:27.665448 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662180 2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662182 2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662185 2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662187 2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662190 2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662193 2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662195 2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662198 2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662200 2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662203 2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662206 2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662208 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662211 2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662214 2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662216 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662220 2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662224 2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662227 2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662230 2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:27.665907 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662233 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662236 2573 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662238 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662241 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662243 2573 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662246 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662249 2573 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662251 2573 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662254 2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662256 2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662259 2573 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662262 2573 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662265 2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662267 2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662270 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662272 2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662274 2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662277 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662280 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662282 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:27.666352 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662285 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662287 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662289 2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662293 2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662296 2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662298 2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662301 2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662304 2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662306 2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662311 2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662314 2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662316 2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662319 2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662321 2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662324 2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662327 2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662329 2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662332 2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662335 2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662337 2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:27.666861 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662340 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662342 2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662345 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662348 2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662351 2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662353 2573 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662356 2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662359 2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662362 2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662365 2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662367 2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662370 2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662372 2573 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662375 2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662377 2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662380 2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662383 2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662386 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662389 2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662392 2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:27.667331 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662394 2573 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662398 2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662401 2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.662404 2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663451 2573 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663461 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663468 2573 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663473 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663477 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663481 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663485 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663489 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663493 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663495 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663500 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663503 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663506 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663510 2573 flags.go:64] FLAG: --cgroup-root="" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663513 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663516 2573 flags.go:64] FLAG: --client-ca-file="" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663518 2573 flags.go:64] FLAG: --cloud-provider="external" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663521 2573 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663525 2573 flags.go:64] FLAG: --cluster-domain="" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663527 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 05:26:27.667832 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663530 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663533 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663537 2573 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663541 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663544 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663548 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663551 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663555 2573 flags.go:64] FLAG: --contention-profiling="false" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663558 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663562 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663565 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663568 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663572 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663575 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663578 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663581 2573 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663584 2573 flags.go:64] FLAG: --enable-server="true" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663587 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663593 2573 flags.go:64] FLAG: --event-burst="100" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663596 2573 flags.go:64] FLAG: --event-qps="50" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663599 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663605 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663608 2573 flags.go:64] FLAG: --eviction-hard="" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663612 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 05:26:27.668385 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663615 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663618 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663621 2573 flags.go:64] FLAG: --eviction-soft="" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663624 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663627 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663630 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663633 2573 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663635 2573 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663638 2573 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663641 2573 flags.go:64] FLAG: --feature-gates="" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663648 2573 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663651 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663654 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663657 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663670 2573 flags.go:64] FLAG: --healthz-port="10248" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663674 2573 flags.go:64] FLAG: --help="false" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663677 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663680 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663684 2573 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663687 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663690 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663694 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663697 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 05:26:27.668997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663700 2573 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663703 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663706 2573 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663708 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663712 2573 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663714 2573 flags.go:64] FLAG: --kube-reserved="" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663719 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663722 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663725 2573 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663727 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663730 2573 flags.go:64] FLAG: --lock-file="" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663733 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663736 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663739 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663744 2573 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663747 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663750 2573 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663753 2573 flags.go:64] FLAG: --logging-format="text" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663756 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663760 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663763 2573 flags.go:64] FLAG: --manifest-url="" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663766 2573 flags.go:64] FLAG: --manifest-url-header="" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663771 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663774 2573 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663778 2573 flags.go:64] FLAG: --max-pods="110" Nov 04 05:26:27.669538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663781 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663785 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663788 2573 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663791 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663794 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663797 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663800 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663807 2573 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663810 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663813 2573 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663816 2573 flags.go:64] FLAG: --pod-cidr="" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663819 2573 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663822 2573 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663827 2573 flags.go:64] FLAG: --pods-per-core="0" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663830 2573 flags.go:64] FLAG: --port="10250" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663833 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663835 2573 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-056c087ce19aea98d" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663839 2573 flags.go:64] FLAG: --qos-reserved="" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663842 2573 flags.go:64] FLAG: --read-only-port="10255" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663844 2573 flags.go:64] FLAG: --register-node="true" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663847 2573 flags.go:64] FLAG: --register-with-taints="" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663851 2573 flags.go:64] FLAG: --registry-burst="10" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663854 2573 flags.go:64] FLAG: --registry-qps="5" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663857 2573 flags.go:64] FLAG: --reserved-cpus="" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663860 2573 flags.go:64] FLAG: --reserved-memory="" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663863 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 05:26:27.670227 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663866 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663869 2573 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663872 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663875 2573 flags.go:64] FLAG: --runonce="false" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663878 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663881 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663885 2573 flags.go:64] FLAG: --seccomp-default="false" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663888 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663891 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663894 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663897 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663900 2573 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663903 2573 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663906 2573 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663909 2573 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663911 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663914 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663917 2573 flags.go:64] FLAG: --system-cgroups="" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663920 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663926 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663929 2573 flags.go:64] FLAG: --tls-cert-file="" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663932 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663935 2573 flags.go:64] FLAG: --tls-min-version="" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663939 2573 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663942 2573 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 05:26:27.671091 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663945 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663948 2573 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663951 2573 flags.go:64] FLAG: --v="2" Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663956 2573 flags.go:64] FLAG: --version="false" Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663960 2573 flags.go:64] FLAG: --vmodule="" Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663963 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.663967 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664140 2573 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664156 2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664160 2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664164 2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664166 2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664170 2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664174 2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664177 2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664180 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664183 2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664186 2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664189 2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664192 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664194 2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:27.671901 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664197 2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664200 2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664202 2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664205 2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664208 2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664211 2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664214 2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664218 2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664223 2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664227 2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664231 2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664234 2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664240 2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664243 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664246 2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664249 2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664253 2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664256 2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664259 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:27.672410 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664263 2573 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664267 2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664271 2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664274 2573 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664278 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664281 2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664283 2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664287 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664290 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664292 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664295 2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664298 2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664301 2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664303 2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664306 2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664309 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664311 2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664314 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664317 2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:27.672905 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664321 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664324 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664327 2573 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664329 2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664332 2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664335 2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664337 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664340 2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664343 2573 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664346 2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664349 2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664352 2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664355 2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664357 2573 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664360 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664363 2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664366 2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664369 2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664372 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664374 2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:27.673346 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664377 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664380 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664382 2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664385 2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664388 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664390 2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664393 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664396 2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664398 2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664401 2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664404 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664407 2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664409 2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664412 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664414 2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664417 2573 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664420 2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664422 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664425 2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664428 2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:27.673851 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664431 2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:27.674315 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664434 2573 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:27.674315 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664437 2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:27.674315 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664442 2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:27.674315 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664445 2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:27.674315 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664447 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:27.674315 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664451 2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:27.674315 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664453 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:27.674315 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664456 2573 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:27.674315 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664458 2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:27.674315 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664461 2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:27.674315 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664464 2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:27.674315 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664466 2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:27.674315 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664469 2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:27.674315 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.664507 2573 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:26:27.674315 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664574 2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:27.674315 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664579 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664583 2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664586 2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664589 2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664592 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664595 2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664598 2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664601 2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664603 2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664606 2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664609 2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664612 2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664614 2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664618 2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664623 2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664627 2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664629 2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664646 2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664650 2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:27.674710 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664653 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664657 2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664660 2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664677 2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664681 2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664686 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664689 2573 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664692 2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664696 2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664699 2573 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664702 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664704 2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664707 2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664709 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664712 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664715 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664717 2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664720 2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664723 2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:27.675153 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664726 2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664728 2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664731 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664734 2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664736 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664739 2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664742 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664744 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664747 2573 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664750 2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664752 2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664755 2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664763 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664766 2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664770 2573 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664773 2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664776 2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664779 2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664781 2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664784 2573 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:27.675620 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664786 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664790 2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664793 2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664796 2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664798 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664801 2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664803 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664806 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664808 2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664811 2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664813 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664816 2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664819 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664822 2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664824 2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664827 2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664829 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664832 2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664834 2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:27.676109 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664837 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664839 2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664842 2573 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664845 2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664847 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664850 2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664859 2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664863 2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664866 2573 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664869 2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664872 2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664874 2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664877 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664880 2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664882 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664885 2573 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664888 2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664890 2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664893 2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664895 2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:27.676557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664898 2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664901 2573 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664903 2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664906 2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664909 2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664911 2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664914 2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.664918 2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.673360 2573 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.673375 2573 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.674005 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.674012 2573 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674079 2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674084 2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674088 2573 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674091 2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674094 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674097 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674100 2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674102 2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:27.677100 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674105 2573 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674108 2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674111 2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674114 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674116 2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674119 2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674122 2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674125 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674127 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674130 2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674133 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674136 2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674139 2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674142 2573 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674144 2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674147 2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674151 2573 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674153 2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674174 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674177 2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:27.677557 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674180 2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674184 2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674187 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674189 2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674192 2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674194 2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674197 2573 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674201 2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674204 2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674207 2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674209 2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674212 2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674215 2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674217 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674220 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674223 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674225 2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674228 2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674231 2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:27.678049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674233 2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674236 2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674239 2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674241 2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674244 2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674248 2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674251 2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674253 2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674256 2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674258 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674261 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674264 2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674266 2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674269 2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674272 2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674276 2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674280 2573 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674283 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674286 2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:27.678507 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674290 2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674292 2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674295 2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674298 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674300 2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674303 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674306 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674308 2573 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674311 2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674313 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674316 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674319 2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674322 2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674326 2573 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674329 2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674332 2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674334 2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674337 2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674340 2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674342 2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:27.678965 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674345 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674347 2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674350 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674352 2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674355 2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674358 2573 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674361 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674363 2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674366 2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674368 2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674371 2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674374 2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674378 2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674382 2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674385 2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674388 2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674390 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674393 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674396 2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:27.679427 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674398 2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:27.679883 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.674428 2573 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:26:27.679883 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674598 2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:27.679883 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674605 2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:27.679883 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674608 2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:27.679883 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674611 2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:27.679883 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674614 2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:27.679883 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674618 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:27.679883 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674620 2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:27.679883 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674623 2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:27.679883 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674625 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:27.679883 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674628 2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:27.679883 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674631 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:27.679883 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674633 2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:27.679883 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674636 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:27.679883 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674638 2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:27.679883 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674641 2573 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674643 2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674646 2573 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674649 2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674651 2573 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674654 2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674656 2573 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674659 2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674676 2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674679 2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674681 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674684 2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674687 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674690 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674692 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674695 2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674697 2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674700 2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674702 2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674705 2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:27.680272 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674707 2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674710 2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674713 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674716 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674720 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674723 2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674726 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674728 2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674731 2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674734 2573 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674736 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674739 2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674741 2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674744 2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674747 2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674749 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674753 2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674755 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674758 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:27.680816 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674760 2573 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674763 2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674765 2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674769 2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674773 2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674777 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674779 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674782 2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674784 2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674787 2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674790 2573 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674792 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674796 2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674800 2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674803 2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674806 2573 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674809 2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674811 2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674814 2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:27.681250 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674817 2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674820 2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674822 2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674825 2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674827 2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674830 2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674832 2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674835 2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674838 2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674840 2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674843 2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674845 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674848 2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674851 2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674853 2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674856 2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674858 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674861 2573 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674864 2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674866 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:27.681844 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674869 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:27.682320 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674871 2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:27.682320 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674874 2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:27.682320 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674876 2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:27.682320 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674879 2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:27.682320 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674882 2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:27.682320 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674884 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:27.682320 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674887 2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:27.682320 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674889 2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:27.682320 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674893 2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:27.682320 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674895 2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:27.682320 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674898 2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:27.682320 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674901 2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:27.682320 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674904 2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:27.682320 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.674927 2573 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:26:27.682320 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674963 2573 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:27.682320 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674967 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674971 2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674974 2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674977 2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674979 2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674982 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674985 2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674987 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674990 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674993 2573 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674995 2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.674998 2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675002 2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675005 2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675008 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675011 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675013 2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675016 2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:27.682715 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675018 2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675021 2573 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675023 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675027 2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675031 2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675034 2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675036 2573 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675039 2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675042 2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675044 2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675047 2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675050 2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675056 2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675058 2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675061 2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675064 2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675066 2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675069 2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675071 2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675074 2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:27.683125 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675077 2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675079 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675082 2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675084 2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675087 2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675089 2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675092 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675095 2573 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675097 2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675100 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675102 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675105 2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675108 2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675110 2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675113 2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675115 2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675118 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675120 2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675123 2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675126 2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:27.683581 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675129 2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675131 2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675134 2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675136 2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675141 2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675144 2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675146 2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675149 2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675152 2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675154 2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675157 2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675160 2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675163 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675165 2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675168 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675171 2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675173 2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675176 2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675178 2573 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675181 2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:27.684083 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675183 2573 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675186 2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675189 2573 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675191 2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675194 2573 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675196 2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675199 2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675201 2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675204 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675207 2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675209 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675212 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675214 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675217 2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675219 2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675222 2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675226 2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675229 2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675231 2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675234 2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:27.684534 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675236 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:27.685016 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675239 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:27.685016 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675242 2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:27.685016 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675244 2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:27.685016 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675248 2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:27.685016 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675250 2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:27.685016 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:27.675253 2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:27.685016 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.675305 2573 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 05:26:27.685016 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.677431 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Nov 04 05:26:27.685016 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.678462 2573 server.go:1014] "Starting client certificate rotation" Nov 04 05:26:27.685016 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.678562 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 05:26:27.685016 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.678596 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 05:26:27.685016 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.680682 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 05:26:27.685016 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.681856 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 05:26:27.696089 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.696073 2573 log.go:25] "Validated CRI v1 runtime API" Nov 04 05:26:27.700140 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.700126 2573 log.go:25] "Validated CRI v1 image API" Nov 04 05:26:27.703335 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.703311 2573 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 05:26:27.708059 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.708036 2573 fs.go:135] Filesystem UUIDs: map[70bf3a10-78bc-441e-a01c-7110aac0c462:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 7fe47483-ccd5-4130-8ca7-e45ccb05ef7c:/dev/nvme0n1p3] Nov 04 05:26:27.708117 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.708058 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Nov 04 05:26:27.715005 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.714884 2573 manager.go:217] Machine: {Timestamp:2026-11-04 05:26:27.713642269 +0000 UTC m=+0.386780372 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095962 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2330219e9eb2fc31eabb1d3abbbfbd SystemUUID:ec233021-9e9e-b2fc-31ea-bb1d3abbbfbd BootID:9a27b69e-26d7-4df4-a738-76bc959f0dcd Filesystems:[{Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:c2:64:f2:6b Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:c2:64:f2:6b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:40:35:b1:3c:8c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 05:26:27.715792 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.715781 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 05:26:27.715874 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.715862 2573 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 05:26:27.716973 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.716945 2573 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 05:26:27.717099 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.716974 2573 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-143.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 05:26:27.717157 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.717101 2573 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 05:26:27.717157 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.717109 2573 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 05:26:27.717157 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.717119 2573 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 05:26:27.717157 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.717134 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 05:26:27.718245 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.718235 2573 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 05:26:27.722740 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.722727 2573 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 05:26:27.722865 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.722856 2573 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 05:26:27.723790 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.723779 2573 kubelet.go:493] "Attempting to sync node with API server" Nov 04 05:26:27.723824 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.723795 2573 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 05:26:27.723824 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.723805 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 05:26:27.723824 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.723814 2573 kubelet.go:405] "Adding apiserver pod source" Nov 04 05:26:27.723824 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.723822 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 05:26:27.724133 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.724122 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 05:26:27.726409 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.726390 2573 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 05:26:27.732691 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.732675 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 05:26:27.734905 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.734890 2573 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 05:26:27.734982 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.734930 2573 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 05:26:27.735142 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.735129 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 05:26:27.735208 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.735150 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 05:26:27.735208 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.735161 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 05:26:27.735208 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.735171 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 05:26:27.735208 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.735180 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 05:26:27.735208 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.735189 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 05:26:27.735835 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.735825 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 05:26:27.735886 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.735838 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 05:26:27.735886 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.735848 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 05:26:27.735886 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.735858 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 05:26:27.735886 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.735872 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 05:26:27.735886 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.735886 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 05:26:27.736878 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.736867 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 05:26:27.736932 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.736881 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 05:26:27.741557 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.741541 2573 server.go:1263] "Started kubelet" Nov 04 05:26:27.741691 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.741640 2573 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 05:26:27.741757 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.741632 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 05:26:27.741799 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.741786 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 05:26:27.742255 ip-10-0-133-143 systemd[1]: Started Kubernetes Kubelet. Nov 04 05:26:27.743455 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.743437 2573 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 05:26:27.743914 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.743876 2573 server.go:317] "Adding debug handlers to kubelet server" Nov 04 05:26:27.747962 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:27.747942 2573 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Nov 04 05:26:27.749960 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.749946 2573 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 05:26:27.750018 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.749965 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 05:26:27.750066 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:27.750031 2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-143.us-west-2.compute.internal\" not found" Nov 04 05:26:27.750581 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.750564 2573 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 05:26:27.750685 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.750585 2573 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 05:26:27.750685 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.750563 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 05:26:27.750685 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.750651 2573 reconstruct.go:97] "Volume reconstruction finished" Nov 04 05:26:27.750685 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.750677 2573 reconciler.go:29] "Reconciler: start to sync state" Nov 04 05:26:27.750944 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.750928 2573 factory.go:55] Registering systemd factory Nov 04 05:26:27.751008 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.750947 2573 factory.go:223] Registration of the systemd container factory successfully Nov 04 05:26:27.751161 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.751142 2573 factory.go:153] Registering CRI-O factory Nov 04 05:26:27.751161 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.751163 2573 factory.go:223] Registration of the crio container factory successfully Nov 04 05:26:27.751297 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.751212 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 05:26:27.751297 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.751231 2573 factory.go:103] Registering Raw factory Nov 04 05:26:27.751297 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.751244 2573 manager.go:1199] Started watching for new ooms in manager Nov 04 05:26:27.752270 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.752253 2573 manager.go:319] Starting recovery of all containers Nov 04 05:26:27.763973 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.763953 2573 manager.go:324] Recovery completed Nov 04 05:26:27.769114 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.769102 2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:27.771322 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.771300 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:27.771373 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.771337 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:27.771373 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.771351 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:27.771790 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.771777 2573 cpu_manager.go:226] "Starting" policy="none" Nov 04 05:26:27.771790 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.771790 2573 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 05:26:27.771874 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.771806 2573 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 05:26:27.771908 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.771783 2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:27.772710 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.772692 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:27.772806 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.772721 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:27.772806 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.772736 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:27.774030 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.774019 2573 policy_none.go:50] "Start" Nov 04 05:26:27.774074 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.774034 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 05:26:27.774074 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.774044 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 05:26:27.776056 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.776041 2573 policy_none.go:44] "Start" Nov 04 05:26:27.800245 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.800216 2573 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 05:26:27.802510 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.802499 2573 manager.go:341] "Starting Device Plugin manager" Nov 04 05:26:27.802595 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:27.802522 2573 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 05:26:27.802595 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.802530 2573 server.go:87] "Starting device plugin registration server" Nov 04 05:26:27.802790 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.802776 2573 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 05:26:27.802863 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.802789 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 05:26:27.802999 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.802884 2573 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 05:26:27.803121 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.803075 2573 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 05:26:27.803121 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.803085 2573 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 05:26:27.803479 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:27.803464 2573 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 05:26:27.803542 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:27.803512 2573 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-143.us-west-2.compute.internal\" not found" Nov 04 05:26:27.879386 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.879365 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 05:26:27.881219 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.880806 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 05:26:27.881219 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.881154 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-143.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 04 05:26:27.881219 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:27.881167 2573 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-143.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Nov 04 05:26:27.881447 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.881428 2573 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 05:26:27.881543 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.881460 2573 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 05:26:27.881543 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:27.881497 2573 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Nov 04 05:26:27.903537 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.903522 2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:27.904230 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.904217 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:27.904295 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.904245 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:27.904295 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.904259 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:27.904295 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.904281 2573 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:27.928003 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:27.927106 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-143.us-west-2.compute.internal.18e4c04cd016f505 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-143.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-143.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 05:26:27.741512965 +0000 UTC m=+0.414651061,LastTimestamp:2026-11-04 05:26:27.741512965 +0000 UTC m=+0.414651061,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-143.us-west-2.compute.internal,}" Nov 04 05:26:27.949833 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.949815 2573 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 05:26:27.949914 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.949852 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xm9b8" Nov 04 05:26:27.982115 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.982094 2573 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal"] Nov 04 05:26:27.982173 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.982142 2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:27.982949 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.982934 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:27.982994 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.982964 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:27.982994 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.982983 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:27.985139 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.985126 2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:27.985289 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.985274 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:27.985329 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.985303 2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:27.985851 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.985830 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:27.985929 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.985857 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:27.985929 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.985869 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:27.985929 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.985889 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:27.985929 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.985898 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:27.985929 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.985872 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:27.987900 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.987887 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:27.987963 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.987908 2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:27.988718 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.988701 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:27.988781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.988728 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:27.988781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:27.988737 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:28.008903 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:28.008876 2573 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-143.us-west-2.compute.internal\" not found" node="ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:28.010953 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:28.010938 2573 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-143.us-west-2.compute.internal\" not found" node="ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:28.036430 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.036392 2573 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:26:28.052166 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.052145 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94f347668688382ceddf8a0c03be4355-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"94f347668688382ceddf8a0c03be4355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:28.052246 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.052172 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94f347668688382ceddf8a0c03be4355-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"94f347668688382ceddf8a0c03be4355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:28.052246 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.052190 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9d7645ed4719bb9894aec65f339b9ca6-config\") pod \"kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"9d7645ed4719bb9894aec65f339b9ca6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:28.083000 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.082974 2573 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:26:28.087678 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.087636 2573 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:26:28.129814 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.129793 2573 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:26:28.130476 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.130462 2573 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:28.130527 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:28.130482 2573 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-143.us-west-2.compute.internal\": node \"ip-10-0-133-143.us-west-2.compute.internal\" not found" Nov 04 05:26:28.150296 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.150274 2573 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:28.152492 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.152477 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94f347668688382ceddf8a0c03be4355-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"94f347668688382ceddf8a0c03be4355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:28.152538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.152502 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9d7645ed4719bb9894aec65f339b9ca6-config\") pod \"kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"9d7645ed4719bb9894aec65f339b9ca6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:28.152538 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.152520 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94f347668688382ceddf8a0c03be4355-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"94f347668688382ceddf8a0c03be4355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:28.152631 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.152574 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94f347668688382ceddf8a0c03be4355-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"94f347668688382ceddf8a0c03be4355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:28.152631 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.152575 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94f347668688382ceddf8a0c03be4355-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"94f347668688382ceddf8a0c03be4355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:28.152631 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.152602 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9d7645ed4719bb9894aec65f339b9ca6-config\") pod \"kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"9d7645ed4719bb9894aec65f339b9ca6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:28.256009 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.255960 2573 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:26:28.256086 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.256046 2573 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:28.314121 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.314103 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:28.316253 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.316236 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:26:28.351658 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.351640 2573 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:26:28.679120 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.679088 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Nov 04 05:26:28.679604 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.679203 2573 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 04 05:26:28.679604 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.679244 2573 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 04 05:26:28.679604 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.679258 2573 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 04 05:26:28.679604 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:28.679256 2573 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a6110e2483d664bf1a253f79dfb04657-a868372cdfab352f.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-133-143.us-west-2.compute.internal.18e4c04cd1ddd1e7\": read tcp 10.0.133.143:34586->18.226.6.180:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-133-143.us-west-2.compute.internal.18e4c04cd1ddd1e7 default 4877 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-143.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-143.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-143.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 05:26:27 +0000 UTC,LastTimestamp:2026-11-04 05:26:27.982948415 +0000 UTC m=+0.656086511,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-143.us-west-2.compute.internal,}" Nov 04 05:26:28.724550 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.724530 2573 apiserver.go:52] "Watching apiserver" Nov 04 05:26:28.750793 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.750774 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Nov 04 05:26:28.955579 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.955544 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 05:21:27 +0000 UTC" deadline="2028-08-20 05:01:23.535920154 +0000 UTC" Nov 04 05:26:28.955579 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.955570 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15719h34m54.580353749s" Nov 04 05:26:28.985214 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.985189 2573 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 05:26:28.985574 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.985520 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-ddzfn","openshift-multus/network-metrics-daemon-55qcs","kube-system/global-pull-secret-syncer-nk2gk","kube-system/konnectivity-agent-x27vz","kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal","openshift-dns/node-resolver-87f78","openshift-image-registry/node-ca-m9v8n","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal","openshift-multus/multus-vmgxh","openshift-network-operator/iptables-alerter-q7mtr","openshift-ovn-kubernetes/ovnkube-node-7tqkj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn","openshift-network-diagnostics/network-check-target-27hf6","openshift-cluster-node-tuning-operator/tuned-wrd6s"] Nov 04 05:26:28.988122 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.988104 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:28.990301 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.990280 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:28.990404 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:28.990341 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55qcs" podUID="4a509700-2c8d-4113-9121-3ec287223f70" Nov 04 05:26:28.992368 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.992353 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:28.992418 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:28.992405 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nk2gk" podUID="26ae8a36-06bf-414e-b2bd-325033800b72" Nov 04 05:26:28.994377 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.994366 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:26:28.996730 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.996713 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-87f78" Nov 04 05:26:28.998880 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:28.998863 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:26:29.001388 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.001371 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.004406 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.004387 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:26:29.006570 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.006557 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.009090 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.009078 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.011338 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.011325 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:29.011398 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.011374 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27hf6" podUID="4a5449c3-4f21-44d2-9d10-9a0956aefdcf" Nov 04 05:26:29.013559 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.013544 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.039472 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.039450 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 05:26:29.039559 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.039456 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 05:26:29.039607 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.039576 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 05:26:29.039607 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.039579 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 05:26:29.039607 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.039591 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qgv9m\"" Nov 04 05:26:29.039702 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.039641 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 05:26:29.052353 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.052330 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 05:26:29.059431 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059412 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qghpk\" (UniqueName: \"kubernetes.io/projected/5e7c38cd-3943-484d-b433-cbec7bce35c6-kube-api-access-qghpk\") pod \"node-ca-m9v8n\" (UID: \"5e7c38cd-3943-484d-b433-cbec7bce35c6\") " pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:26:29.059514 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059443 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-ovnkube-script-lib\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.059514 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059496 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-modprobe-d\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.059587 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059531 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-system-cni-dir\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.059587 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059557 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-k8s-cni-cncf-io\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.059648 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059590 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-netns\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.059648 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059615 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-kubelet\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.059648 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059631 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77f9e498-8338-4f89-ae13-e72e0108a602-multus-daemon-config\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.059781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059649 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysctl-d\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.059781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059666 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-systemd\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.059781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059699 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-registration-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.059781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059716 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpww4\" (UniqueName: \"kubernetes.io/projected/38b751da-304f-4000-bf56-85e1d9aa8152-kube-api-access-cpww4\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.059781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059739 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbx8t\" (UniqueName: \"kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t\") pod \"network-check-target-27hf6\" (UID: \"4a5449c3-4f21-44d2-9d10-9a0956aefdcf\") " pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:29.059781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059767 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-484lx\" (UniqueName: \"kubernetes.io/projected/4a509700-2c8d-4113-9121-3ec287223f70-kube-api-access-484lx\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:29.059974 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059787 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-ovn\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.059974 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059802 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trzdq\" (UniqueName: \"kubernetes.io/projected/72dd8eb9-265d-4761-a507-d91e29606698-kube-api-access-trzdq\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.059974 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059821 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.059974 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059883 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-socket-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.059974 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059916 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-cni-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.059974 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059931 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-cnibin\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.059974 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059946 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-socket-dir-parent\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.059974 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059961 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-slash\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.060293 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.059975 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysctl-conf\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.060293 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060009 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:29.060293 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060037 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/428ba86e-0dd8-4a0a-b899-c7e9d568487c-tmp-dir\") pod \"node-resolver-87f78\" (UID: \"428ba86e-0dd8-4a0a-b899-c7e9d568487c\") " pod="openshift-dns/node-resolver-87f78" Nov 04 05:26:29.060293 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060051 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-etc-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.060293 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060101 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-run-ovn-kubernetes\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.060293 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060139 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.060293 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060185 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.060293 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060204 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.060293 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060245 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x87gt\" (UniqueName: \"kubernetes.io/projected/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-kube-api-access-x87gt\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.060293 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060264 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-hostroot\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.060293 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060285 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-system-cni-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060346 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-sys-fs\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060378 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-lib-modules\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060423 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-host\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060451 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f7d857f-84f9-4bde-9b5a-66164c90377e-tmp\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060468 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-multus-certs\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060493 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060524 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77f9e498-8338-4f89-ae13-e72e0108a602-cni-binary-copy\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060546 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-cni-multus\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060560 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/428ba86e-0dd8-4a0a-b899-c7e9d568487c-hosts-file\") pod \"node-resolver-87f78\" (UID: \"428ba86e-0dd8-4a0a-b899-c7e9d568487c\") " pod="openshift-dns/node-resolver-87f78" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060575 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e7c38cd-3943-484d-b433-cbec7bce35c6-host\") pod \"node-ca-m9v8n\" (UID: \"5e7c38cd-3943-484d-b433-cbec7bce35c6\") " pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060603 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e7c38cd-3943-484d-b433-cbec7bce35c6-serviceca\") pod \"node-ca-m9v8n\" (UID: \"5e7c38cd-3943-484d-b433-cbec7bce35c6\") " pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060620 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cnibin\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060634 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060652 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-etc-selinux\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060691 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-kubelet\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060725 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-systemd-units\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.060852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060753 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-run\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.061566 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060776 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjmwm\" (UniqueName: \"kubernetes.io/projected/8f7d857f-84f9-4bde-9b5a-66164c90377e-kube-api-access-hjmwm\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.061566 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060801 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-conf-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.061566 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060826 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.061566 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060854 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-cni-netd\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.061566 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060877 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-sys\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.061566 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060932 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-run-netns\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.061566 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.060970 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-cni-bin\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.061566 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061001 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-kubernetes\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.061566 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061026 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d5fj\" (UniqueName: \"kubernetes.io/projected/6cf77b4d-32f5-42db-851f-3466ecf583c9-kube-api-access-8d5fj\") pod \"iptables-alerter-q7mtr\" (UID: \"6cf77b4d-32f5-42db-851f-3466ecf583c9\") " pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:26:29.061566 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061059 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-device-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.061566 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061084 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6cf77b4d-32f5-42db-851f-3466ecf583c9-iptables-alerter-script\") pod \"iptables-alerter-q7mtr\" (UID: \"6cf77b4d-32f5-42db-851f-3466ecf583c9\") " pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:26:29.061566 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061111 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-log-socket\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.061566 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061127 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-env-overrides\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.061566 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061142 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.061566 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061158 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/26ae8a36-06bf-414e-b2bd-325033800b72-dbus\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:29.061566 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061182 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-cni-bin\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.062135 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061207 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cf77b4d-32f5-42db-851f-3466ecf583c9-host-slash\") pod \"iptables-alerter-q7mtr\" (UID: \"6cf77b4d-32f5-42db-851f-3466ecf583c9\") " pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:26:29.062135 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061232 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a566754-7023-408d-bb6a-3eb984ffa7e3-konnectivity-ca\") pod \"konnectivity-agent-x27vz\" (UID: \"1a566754-7023-408d-bb6a-3eb984ffa7e3\") " pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:26:29.062135 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061258 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-var-lib-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.062135 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061273 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-ovnkube-config\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.062135 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061308 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysconfig\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.062135 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061340 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-var-lib-kubelet\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.062135 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061365 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/26ae8a36-06bf-414e-b2bd-325033800b72-kubelet-config\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:29.062135 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061389 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-node-log\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.062135 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061415 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72dd8eb9-265d-4761-a507-d91e29606698-ovn-node-metrics-cert\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.062135 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061454 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a566754-7023-408d-bb6a-3eb984ffa7e3-agent-certs\") pod \"konnectivity-agent-x27vz\" (UID: \"1a566754-7023-408d-bb6a-3eb984ffa7e3\") " pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:26:29.062135 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061481 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-systemd\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.062135 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061501 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-tuned\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.062135 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061523 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-etc-kubernetes\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.062135 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061546 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qw4k\" (UniqueName: \"kubernetes.io/projected/77f9e498-8338-4f89-ae13-e72e0108a602-kube-api-access-2qw4k\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.062135 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.061562 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s7c8\" (UniqueName: \"kubernetes.io/projected/428ba86e-0dd8-4a0a-b899-c7e9d568487c-kube-api-access-9s7c8\") pod \"node-resolver-87f78\" (UID: \"428ba86e-0dd8-4a0a-b899-c7e9d568487c\") " pod="openshift-dns/node-resolver-87f78" Nov 04 05:26:29.142826 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.142807 2573 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 05:26:29.142910 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.142848 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-z7g2c" Nov 04 05:26:29.145789 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.145770 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 05:26:29.146023 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.146010 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 05:26:29.146084 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.146011 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 05:26:29.146084 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.146051 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 05:26:29.146179 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.146098 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vw645\"" Nov 04 05:26:29.146179 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.146156 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 05:26:29.146315 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.146294 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 05:26:29.146315 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.146294 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:26:29.146462 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.146310 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 05:26:29.146462 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.146440 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 05:26:29.146539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.146490 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 05:26:29.146539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.146507 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z6mlw\"" Nov 04 05:26:29.160721 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.160697 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 05:26:29.160826 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.160758 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 05:26:29.160826 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.160778 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 05:26:29.160826 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.160803 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hqws2\"" Nov 04 05:26:29.160976 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.160893 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 05:26:29.160976 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.160920 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 05:26:29.160976 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.160960 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:26:29.161084 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.160959 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fz9sj\"" Nov 04 05:26:29.161084 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.161033 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 05:26:29.161290 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.161264 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 05:26:29.161290 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.161268 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 05:26:29.161419 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.161295 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 05:26:29.161454 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.161423 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 05:26:29.162417 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162399 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-run-netns\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.162455 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162427 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-cni-bin\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.162455 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162442 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-kubernetes\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.162520 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162459 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d5fj\" (UniqueName: \"kubernetes.io/projected/6cf77b4d-32f5-42db-851f-3466ecf583c9-kube-api-access-8d5fj\") pod \"iptables-alerter-q7mtr\" (UID: \"6cf77b4d-32f5-42db-851f-3466ecf583c9\") " pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:26:29.162520 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162499 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-kubernetes\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.162520 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162501 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-cni-bin\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.162605 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162502 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-run-netns\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.162605 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162522 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-device-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.162605 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162539 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-device-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.162605 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162550 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6cf77b4d-32f5-42db-851f-3466ecf583c9-iptables-alerter-script\") pod \"iptables-alerter-q7mtr\" (UID: \"6cf77b4d-32f5-42db-851f-3466ecf583c9\") " pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:26:29.162605 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162568 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-log-socket\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.162605 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162595 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-env-overrides\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.162909 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162623 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.162909 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162634 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-log-socket\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.162909 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162649 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/26ae8a36-06bf-414e-b2bd-325033800b72-dbus\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:29.162909 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162735 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/26ae8a36-06bf-414e-b2bd-325033800b72-dbus\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:29.162909 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162767 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-cni-bin\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.162909 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162792 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cf77b4d-32f5-42db-851f-3466ecf583c9-host-slash\") pod \"iptables-alerter-q7mtr\" (UID: \"6cf77b4d-32f5-42db-851f-3466ecf583c9\") " pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:26:29.162909 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162816 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a566754-7023-408d-bb6a-3eb984ffa7e3-konnectivity-ca\") pod \"konnectivity-agent-x27vz\" (UID: \"1a566754-7023-408d-bb6a-3eb984ffa7e3\") " pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:26:29.162909 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162835 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.162909 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162846 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-var-lib-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.162909 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162857 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cf77b4d-32f5-42db-851f-3466ecf583c9-host-slash\") pod \"iptables-alerter-q7mtr\" (UID: \"6cf77b4d-32f5-42db-851f-3466ecf583c9\") " pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:26:29.162909 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162821 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-cni-bin\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.162909 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162891 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-var-lib-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.162909 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162898 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-ovnkube-config\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162928 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysconfig\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162956 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-var-lib-kubelet\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.162982 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/26ae8a36-06bf-414e-b2bd-325033800b72-kubelet-config\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163008 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-node-log\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163021 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysconfig\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163034 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72dd8eb9-265d-4761-a507-d91e29606698-ovn-node-metrics-cert\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163066 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a566754-7023-408d-bb6a-3eb984ffa7e3-agent-certs\") pod \"konnectivity-agent-x27vz\" (UID: \"1a566754-7023-408d-bb6a-3eb984ffa7e3\") " pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163077 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/26ae8a36-06bf-414e-b2bd-325033800b72-kubelet-config\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163080 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-var-lib-kubelet\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163089 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-env-overrides\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163135 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-node-log\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163090 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-systemd\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163164 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6cf77b4d-32f5-42db-851f-3466ecf583c9-iptables-alerter-script\") pod \"iptables-alerter-q7mtr\" (UID: \"6cf77b4d-32f5-42db-851f-3466ecf583c9\") " pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163177 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-tuned\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163150 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-systemd\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163202 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-etc-kubernetes\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163231 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qw4k\" (UniqueName: \"kubernetes.io/projected/77f9e498-8338-4f89-ae13-e72e0108a602-kube-api-access-2qw4k\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.163539 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163247 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-etc-kubernetes\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163259 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s7c8\" (UniqueName: \"kubernetes.io/projected/428ba86e-0dd8-4a0a-b899-c7e9d568487c-kube-api-access-9s7c8\") pod \"node-resolver-87f78\" (UID: \"428ba86e-0dd8-4a0a-b899-c7e9d568487c\") " pod="openshift-dns/node-resolver-87f78" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163284 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qghpk\" (UniqueName: \"kubernetes.io/projected/5e7c38cd-3943-484d-b433-cbec7bce35c6-kube-api-access-qghpk\") pod \"node-ca-m9v8n\" (UID: \"5e7c38cd-3943-484d-b433-cbec7bce35c6\") " pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163307 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-ovnkube-script-lib\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163329 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-modprobe-d\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163354 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-system-cni-dir\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163366 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-ovnkube-config\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163380 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-k8s-cni-cncf-io\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163395 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a566754-7023-408d-bb6a-3eb984ffa7e3-konnectivity-ca\") pod \"konnectivity-agent-x27vz\" (UID: \"1a566754-7023-408d-bb6a-3eb984ffa7e3\") " pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163381 2573 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163433 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-k8s-cni-cncf-io\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163453 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-system-cni-dir\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163461 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-netns\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163496 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-modprobe-d\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163499 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-kubelet\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163531 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-kubelet\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163534 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77f9e498-8338-4f89-ae13-e72e0108a602-multus-daemon-config\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163556 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysctl-d\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.164449 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163558 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-netns\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163574 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-systemd\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163599 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-registration-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163623 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpww4\" (UniqueName: \"kubernetes.io/projected/38b751da-304f-4000-bf56-85e1d9aa8152-kube-api-access-cpww4\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163645 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-systemd\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163646 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysctl-d\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163648 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbx8t\" (UniqueName: \"kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t\") pod \"network-check-target-27hf6\" (UID: \"4a5449c3-4f21-44d2-9d10-9a0956aefdcf\") " pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163706 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-484lx\" (UniqueName: \"kubernetes.io/projected/4a509700-2c8d-4113-9121-3ec287223f70-kube-api-access-484lx\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163750 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-ovn\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163779 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trzdq\" (UniqueName: \"kubernetes.io/projected/72dd8eb9-265d-4761-a507-d91e29606698-kube-api-access-trzdq\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163804 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163713 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-registration-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163832 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-ovn\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163830 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-socket-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163885 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-cni-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163899 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163912 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-cnibin\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.165257 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163936 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-socket-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163936 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-socket-dir-parent\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163977 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-slash\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.163997 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-socket-dir-parent\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164011 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysctl-conf\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164035 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164049 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-cni-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164063 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77f9e498-8338-4f89-ae13-e72e0108a602-multus-daemon-config\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164081 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-slash\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164053 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-cnibin\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164055 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/428ba86e-0dd8-4a0a-b899-c7e9d568487c-tmp-dir\") pod \"node-resolver-87f78\" (UID: \"428ba86e-0dd8-4a0a-b899-c7e9d568487c\") " pod="openshift-dns/node-resolver-87f78" Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.164142 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164155 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysctl-conf\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164157 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-etc-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164183 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-etc-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.164218 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs podName:4a509700-2c8d-4113-9121-3ec287223f70 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:29.664183617 +0000 UTC m=+2.337321722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs") pod "network-metrics-daemon-55qcs" (UID: "4a509700-2c8d-4113-9121-3ec287223f70") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164232 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-run-ovn-kubernetes\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.165865 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164259 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-run-ovn-kubernetes\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164260 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164281 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164289 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/428ba86e-0dd8-4a0a-b899-c7e9d568487c-tmp-dir\") pod \"node-resolver-87f78\" (UID: \"428ba86e-0dd8-4a0a-b899-c7e9d568487c\") " pod="openshift-dns/node-resolver-87f78" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164291 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164363 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164389 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x87gt\" (UniqueName: \"kubernetes.io/projected/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-kube-api-access-x87gt\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164447 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-hostroot\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164467 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-system-cni-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164483 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-sys-fs\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164505 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-lib-modules\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164529 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-host\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164536 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-system-cni-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164551 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f7d857f-84f9-4bde-9b5a-66164c90377e-tmp\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164577 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-multus-certs\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164606 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164632 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77f9e498-8338-4f89-ae13-e72e0108a602-cni-binary-copy\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.166750 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164648 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-sys-fs\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164662 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-cni-multus\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164681 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-host\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164701 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/428ba86e-0dd8-4a0a-b899-c7e9d568487c-hosts-file\") pod \"node-resolver-87f78\" (UID: \"428ba86e-0dd8-4a0a-b899-c7e9d568487c\") " pod="openshift-dns/node-resolver-87f78" Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164725 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e7c38cd-3943-484d-b433-cbec7bce35c6-host\") pod \"node-ca-m9v8n\" (UID: \"5e7c38cd-3943-484d-b433-cbec7bce35c6\") " pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164749 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e7c38cd-3943-484d-b433-cbec7bce35c6-serviceca\") pod \"node-ca-m9v8n\" (UID: \"5e7c38cd-3943-484d-b433-cbec7bce35c6\") " pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164753 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-lib-modules\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164773 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cnibin\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164796 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164846 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e7c38cd-3943-484d-b433-cbec7bce35c6-host\") pod \"node-ca-m9v8n\" (UID: \"5e7c38cd-3943-484d-b433-cbec7bce35c6\") " pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164862 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/428ba86e-0dd8-4a0a-b899-c7e9d568487c-hosts-file\") pod \"node-resolver-87f78\" (UID: \"428ba86e-0dd8-4a0a-b899-c7e9d568487c\") " pod="openshift-dns/node-resolver-87f78" Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.164869 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164890 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164902 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cnibin\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.164914 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret podName:26ae8a36-06bf-414e-b2bd-325033800b72 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:29.664900719 +0000 UTC m=+2.338038803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret") pod "global-pull-secret-syncer-nk2gk" (UID: "26ae8a36-06bf-414e-b2bd-325033800b72") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164800 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164942 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-etc-selinux\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.167600 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164966 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-kubelet\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164989 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-systemd-units\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.165013 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-run\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164567 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-hostroot\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.165039 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjmwm\" (UniqueName: \"kubernetes.io/projected/8f7d857f-84f9-4bde-9b5a-66164c90377e-kube-api-access-hjmwm\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.164802 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-multus-certs\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.165063 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-conf-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.165088 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.165114 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-cni-multus\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.165122 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-cni-netd\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.165147 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-sys\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.166118 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77f9e498-8338-4f89-ae13-e72e0108a602-cni-binary-copy\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.166710 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.167062 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e7c38cd-3943-484d-b433-cbec7bce35c6-serviceca\") pod \"node-ca-m9v8n\" (UID: \"5e7c38cd-3943-484d-b433-cbec7bce35c6\") " pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.167263 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-systemd-units\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.167262 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.167318 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-conf-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.167330 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-etc-selinux\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.168381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.167371 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-run\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.169240 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.167395 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-kubelet\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.169240 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.167428 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-cni-netd\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.169240 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.167554 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-sys\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.170419 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.170395 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f7d857f-84f9-4bde-9b5a-66164c90377e-tmp\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.170526 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.170430 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-tuned\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.170526 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.170496 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72dd8eb9-265d-4761-a507-d91e29606698-ovn-node-metrics-cert\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.170616 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.170598 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a566754-7023-408d-bb6a-3eb984ffa7e3-agent-certs\") pod \"konnectivity-agent-x27vz\" (UID: \"1a566754-7023-408d-bb6a-3eb984ffa7e3\") " pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:26:29.194834 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.194812 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-8lkk9\"" Nov 04 05:26:29.194834 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.194820 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r57nx\"" Nov 04 05:26:29.194976 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.194837 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-x7j27\"" Nov 04 05:26:29.194976 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.194842 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6nvfw\"" Nov 04 05:26:29.194976 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.194903 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 05:26:29.204767 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.204747 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-ovnkube-script-lib\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.224169 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.224110 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:29.224169 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.224133 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:29.224169 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.224147 2573 projected.go:196] Error preparing data for projected volume kube-api-access-fbx8t for pod openshift-network-diagnostics/network-check-target-27hf6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:29.224355 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.224209 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t podName:4a5449c3-4f21-44d2-9d10-9a0956aefdcf nodeName:}" failed. No retries permitted until 2026-11-04 05:26:29.724191714 +0000 UTC m=+2.397329819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fbx8t" (UniqueName: "kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t") pod "network-check-target-27hf6" (UID: "4a5449c3-4f21-44d2-9d10-9a0956aefdcf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:29.227029 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.227007 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjmwm\" (UniqueName: \"kubernetes.io/projected/8f7d857f-84f9-4bde-9b5a-66164c90377e-kube-api-access-hjmwm\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.227175 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.227155 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qw4k\" (UniqueName: \"kubernetes.io/projected/77f9e498-8338-4f89-ae13-e72e0108a602-kube-api-access-2qw4k\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.227218 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.227164 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-484lx\" (UniqueName: \"kubernetes.io/projected/4a509700-2c8d-4113-9121-3ec287223f70-kube-api-access-484lx\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:29.227289 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.227268 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qghpk\" (UniqueName: \"kubernetes.io/projected/5e7c38cd-3943-484d-b433-cbec7bce35c6-kube-api-access-qghpk\") pod \"node-ca-m9v8n\" (UID: \"5e7c38cd-3943-484d-b433-cbec7bce35c6\") " pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:26:29.227322 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.227290 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s7c8\" (UniqueName: \"kubernetes.io/projected/428ba86e-0dd8-4a0a-b899-c7e9d568487c-kube-api-access-9s7c8\") pod \"node-resolver-87f78\" (UID: \"428ba86e-0dd8-4a0a-b899-c7e9d568487c\") " pod="openshift-dns/node-resolver-87f78" Nov 04 05:26:29.227322 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.227302 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d5fj\" (UniqueName: \"kubernetes.io/projected/6cf77b4d-32f5-42db-851f-3466ecf583c9-kube-api-access-8d5fj\") pod \"iptables-alerter-q7mtr\" (UID: \"6cf77b4d-32f5-42db-851f-3466ecf583c9\") " pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:26:29.227741 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.227706 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x87gt\" (UniqueName: \"kubernetes.io/projected/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-kube-api-access-x87gt\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.227981 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.227960 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trzdq\" (UniqueName: \"kubernetes.io/projected/72dd8eb9-265d-4761-a507-d91e29606698-kube-api-access-trzdq\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.229285 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.229267 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpww4\" (UniqueName: \"kubernetes.io/projected/38b751da-304f-4000-bf56-85e1d9aa8152-kube-api-access-cpww4\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.333713 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.333681 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:26:29.341763 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.341741 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:26:29.364336 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.364313 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-87f78" Nov 04 05:26:29.386739 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.386718 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:26:29.399615 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.399595 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vmgxh" Nov 04 05:26:29.413234 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.413203 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:26:29.433789 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.433760 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:29.440991 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.440469 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:26:29.442823 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.442802 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:26:29.546999 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:29.546967 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d7645ed4719bb9894aec65f339b9ca6.slice/crio-463c38893c854b23e9be888f730e7a64ecaba26981bc47571ba19fa0317bc9fa WatchSource:0}: Error finding container 463c38893c854b23e9be888f730e7a64ecaba26981bc47571ba19fa0317bc9fa: Status 404 returned error can't find the container with id 463c38893c854b23e9be888f730e7a64ecaba26981bc47571ba19fa0317bc9fa Nov 04 05:26:29.547509 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:29.547486 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94f347668688382ceddf8a0c03be4355.slice/crio-e44c5924347099be8c386a1c359936f6810d6c950da0b78944869ce8d5e90a1c WatchSource:0}: Error finding container e44c5924347099be8c386a1c359936f6810d6c950da0b78944869ce8d5e90a1c: Status 404 returned error can't find the container with id e44c5924347099be8c386a1c359936f6810d6c950da0b78944869ce8d5e90a1c Nov 04 05:26:29.552869 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.552854 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:26:29.669282 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.669256 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:29.669415 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.669287 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:29.669415 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.669383 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:29.669481 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.669424 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret podName:26ae8a36-06bf-414e-b2bd-325033800b72 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:30.669412846 +0000 UTC m=+3.342550942 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret") pod "global-pull-secret-syncer-nk2gk" (UID: "26ae8a36-06bf-414e-b2bd-325033800b72") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:29.669481 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.669385 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:29.669481 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.669451 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs podName:4a509700-2c8d-4113-9121-3ec287223f70 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:30.669444596 +0000 UTC m=+3.342582692 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs") pod "network-metrics-daemon-55qcs" (UID: "4a509700-2c8d-4113-9121-3ec287223f70") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:29.770164 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.770136 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbx8t\" (UniqueName: \"kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t\") pod \"network-check-target-27hf6\" (UID: \"4a5449c3-4f21-44d2-9d10-9a0956aefdcf\") " pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:29.770598 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.770256 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:29.770598 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.770268 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:29.770598 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.770276 2573 projected.go:196] Error preparing data for projected volume kube-api-access-fbx8t for pod openshift-network-diagnostics/network-check-target-27hf6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:29.770598 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:29.770322 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t podName:4a5449c3-4f21-44d2-9d10-9a0956aefdcf nodeName:}" failed. No retries permitted until 2026-11-04 05:26:30.770310475 +0000 UTC m=+3.443448557 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbx8t" (UniqueName: "kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t") pod "network-check-target-27hf6" (UID: "4a5449c3-4f21-44d2-9d10-9a0956aefdcf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:29.885473 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.885395 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" event={"ID":"9d7645ed4719bb9894aec65f339b9ca6","Type":"ContainerStarted","Data":"463c38893c854b23e9be888f730e7a64ecaba26981bc47571ba19fa0317bc9fa"} Nov 04 05:26:29.886373 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:29.886357 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" event={"ID":"94f347668688382ceddf8a0c03be4355","Type":"ContainerStarted","Data":"e44c5924347099be8c386a1c359936f6810d6c950da0b78944869ce8d5e90a1c"} Nov 04 05:26:30.110165 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.110137 2573 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:26:30.145800 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.145729 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 05:21:28 +0000 UTC" deadline="2028-08-20 01:07:06.7745688 +0000 UTC" Nov 04 05:26:30.145800 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.145756 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15715h40m36.628817191s" Nov 04 05:26:30.165472 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.165452 2573 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:26:30.250070 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:30.250042 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a566754_7023_408d_bb6a_3eb984ffa7e3.slice/crio-beca5780d1576f0b8b961482c174af7488c9be39a7136b68b4fad07dfa5673db WatchSource:0}: Error finding container beca5780d1576f0b8b961482c174af7488c9be39a7136b68b4fad07dfa5673db: Status 404 returned error can't find the container with id beca5780d1576f0b8b961482c174af7488c9be39a7136b68b4fad07dfa5673db Nov 04 05:26:30.250831 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:30.250804 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cf77b4d_32f5_42db_851f_3466ecf583c9.slice/crio-c8bdb05cd3620d33090b4d863e9485be49c1e5fb7fc1bf8122b59451e19b15e4 WatchSource:0}: Error finding container c8bdb05cd3620d33090b4d863e9485be49c1e5fb7fc1bf8122b59451e19b15e4: Status 404 returned error can't find the container with id c8bdb05cd3620d33090b4d863e9485be49c1e5fb7fc1bf8122b59451e19b15e4 Nov 04 05:26:30.269222 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:30.269198 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef5a3412_cd67_424f_bb1f_1ec122d5e3eb.slice/crio-c65039c243a532bb0b419e22a11d20186c65571e11c590f0f61540e727812975 WatchSource:0}: Error finding container c65039c243a532bb0b419e22a11d20186c65571e11c590f0f61540e727812975: Status 404 returned error can't find the container with id c65039c243a532bb0b419e22a11d20186c65571e11c590f0f61540e727812975 Nov 04 05:26:30.287252 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:30.287234 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77f9e498_8338_4f89_ae13_e72e0108a602.slice/crio-788d9f6ddbe47b4b03358ed63dc914669c3f40a5b8023bb26a91a0681452fddc WatchSource:0}: Error finding container 788d9f6ddbe47b4b03358ed63dc914669c3f40a5b8023bb26a91a0681452fddc: Status 404 returned error can't find the container with id 788d9f6ddbe47b4b03358ed63dc914669c3f40a5b8023bb26a91a0681452fddc Nov 04 05:26:30.294959 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.294943 2573 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:26:30.321819 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:30.321799 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod428ba86e_0dd8_4a0a_b899_c7e9d568487c.slice/crio-0036c1fcf22adf4a202a1b3d09c1f9e85b4fa8d40e0ff508318537d5859f6499 WatchSource:0}: Error finding container 0036c1fcf22adf4a202a1b3d09c1f9e85b4fa8d40e0ff508318537d5859f6499: Status 404 returned error can't find the container with id 0036c1fcf22adf4a202a1b3d09c1f9e85b4fa8d40e0ff508318537d5859f6499 Nov 04 05:26:30.341341 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:30.341322 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e7c38cd_3943_484d_b433_cbec7bce35c6.slice/crio-03b38d7f8d93e469a723b47425c44e0922c67780ce06bb599cf5ee5089fd17ca WatchSource:0}: Error finding container 03b38d7f8d93e469a723b47425c44e0922c67780ce06bb599cf5ee5089fd17ca: Status 404 returned error can't find the container with id 03b38d7f8d93e469a723b47425c44e0922c67780ce06bb599cf5ee5089fd17ca Nov 04 05:26:30.411993 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:30.411974 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f7d857f_84f9_4bde_9b5a_66164c90377e.slice/crio-66817b31a0768fd24bf604f75f52e49b61c1a41fb8a369668b4c814e6e4eaa56 WatchSource:0}: Error finding container 66817b31a0768fd24bf604f75f52e49b61c1a41fb8a369668b4c814e6e4eaa56: Status 404 returned error can't find the container with id 66817b31a0768fd24bf604f75f52e49b61c1a41fb8a369668b4c814e6e4eaa56 Nov 04 05:26:30.412809 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:30.412790 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38b751da_304f_4000_bf56_85e1d9aa8152.slice/crio-3f92655005734c04835a73a0a2c46997d2abe0b481a112c6395733a7731f21cd WatchSource:0}: Error finding container 3f92655005734c04835a73a0a2c46997d2abe0b481a112c6395733a7731f21cd: Status 404 returned error can't find the container with id 3f92655005734c04835a73a0a2c46997d2abe0b481a112c6395733a7731f21cd Nov 04 05:26:30.462275 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:30.462257 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72dd8eb9_265d_4761_a507_d91e29606698.slice/crio-8e46f3b116be4a32a514c6bd4dfe03c8f7a60ce85d4a42a1947d7652e289f25c WatchSource:0}: Error finding container 8e46f3b116be4a32a514c6bd4dfe03c8f7a60ce85d4a42a1947d7652e289f25c: Status 404 returned error can't find the container with id 8e46f3b116be4a32a514c6bd4dfe03c8f7a60ce85d4a42a1947d7652e289f25c Nov 04 05:26:30.678943 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.678915 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:30.679107 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.678948 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:30.679107 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:30.679031 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:30.679107 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:30.679050 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:30.679107 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:30.679079 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret podName:26ae8a36-06bf-414e-b2bd-325033800b72 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:32.679064691 +0000 UTC m=+5.352202790 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret") pod "global-pull-secret-syncer-nk2gk" (UID: "26ae8a36-06bf-414e-b2bd-325033800b72") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:30.679107 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:30.679100 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs podName:4a509700-2c8d-4113-9121-3ec287223f70 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:32.679085371 +0000 UTC m=+5.352223472 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs") pod "network-metrics-daemon-55qcs" (UID: "4a509700-2c8d-4113-9121-3ec287223f70") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:30.779533 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.779495 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbx8t\" (UniqueName: \"kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t\") pod \"network-check-target-27hf6\" (UID: \"4a5449c3-4f21-44d2-9d10-9a0956aefdcf\") " pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:30.779999 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:30.779687 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:30.779999 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:30.779706 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:30.779999 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:30.779715 2573 projected.go:196] Error preparing data for projected volume kube-api-access-fbx8t for pod openshift-network-diagnostics/network-check-target-27hf6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:30.779999 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:30.779760 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t podName:4a5449c3-4f21-44d2-9d10-9a0956aefdcf nodeName:}" failed. No retries permitted until 2026-11-04 05:26:32.779747957 +0000 UTC m=+5.452886066 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbx8t" (UniqueName: "kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t") pod "network-check-target-27hf6" (UID: "4a5449c3-4f21-44d2-9d10-9a0956aefdcf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:30.882146 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.882114 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:30.882270 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.882182 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:30.882322 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.882267 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:30.882322 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:30.882285 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55qcs" podUID="4a509700-2c8d-4113-9121-3ec287223f70" Nov 04 05:26:30.882386 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:30.882345 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nk2gk" podUID="26ae8a36-06bf-414e-b2bd-325033800b72" Nov 04 05:26:30.882386 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:30.882368 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27hf6" podUID="4a5449c3-4f21-44d2-9d10-9a0956aefdcf" Nov 04 05:26:30.888764 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.888724 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vmgxh" event={"ID":"77f9e498-8338-4f89-ae13-e72e0108a602","Type":"ContainerStarted","Data":"788d9f6ddbe47b4b03358ed63dc914669c3f40a5b8023bb26a91a0681452fddc"} Nov 04 05:26:30.889840 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.889819 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q7mtr" event={"ID":"6cf77b4d-32f5-42db-851f-3466ecf583c9","Type":"ContainerStarted","Data":"c8bdb05cd3620d33090b4d863e9485be49c1e5fb7fc1bf8122b59451e19b15e4"} Nov 04 05:26:30.890942 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.890906 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x27vz" event={"ID":"1a566754-7023-408d-bb6a-3eb984ffa7e3","Type":"ContainerStarted","Data":"beca5780d1576f0b8b961482c174af7488c9be39a7136b68b4fad07dfa5673db"} Nov 04 05:26:30.892156 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.892134 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" event={"ID":"38b751da-304f-4000-bf56-85e1d9aa8152","Type":"ContainerStarted","Data":"3f92655005734c04835a73a0a2c46997d2abe0b481a112c6395733a7731f21cd"} Nov 04 05:26:30.893341 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.893321 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" event={"ID":"8f7d857f-84f9-4bde-9b5a-66164c90377e","Type":"ContainerStarted","Data":"66817b31a0768fd24bf604f75f52e49b61c1a41fb8a369668b4c814e6e4eaa56"} Nov 04 05:26:30.894445 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.894426 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m9v8n" event={"ID":"5e7c38cd-3943-484d-b433-cbec7bce35c6","Type":"ContainerStarted","Data":"03b38d7f8d93e469a723b47425c44e0922c67780ce06bb599cf5ee5089fd17ca"} Nov 04 05:26:30.895388 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.895360 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-87f78" event={"ID":"428ba86e-0dd8-4a0a-b899-c7e9d568487c","Type":"ContainerStarted","Data":"0036c1fcf22adf4a202a1b3d09c1f9e85b4fa8d40e0ff508318537d5859f6499"} Nov 04 05:26:30.896382 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.896364 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddzfn" event={"ID":"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb","Type":"ContainerStarted","Data":"c65039c243a532bb0b419e22a11d20186c65571e11c590f0f61540e727812975"} Nov 04 05:26:30.897353 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:30.897336 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" event={"ID":"72dd8eb9-265d-4761-a507-d91e29606698","Type":"ContainerStarted","Data":"8e46f3b116be4a32a514c6bd4dfe03c8f7a60ce85d4a42a1947d7652e289f25c"} Nov 04 05:26:31.145975 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:31.145870 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 05:21:28 +0000 UTC" deadline="2028-06-11 16:00:57.457237122 +0000 UTC" Nov 04 05:26:31.145975 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:31.145923 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14050h34m26.311317835s" Nov 04 05:26:32.698387 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:32.698355 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:32.699012 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:32.698409 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:32.699012 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:32.698549 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:32.699012 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:32.698614 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret podName:26ae8a36-06bf-414e-b2bd-325033800b72 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:36.69859699 +0000 UTC m=+9.371735084 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret") pod "global-pull-secret-syncer-nk2gk" (UID: "26ae8a36-06bf-414e-b2bd-325033800b72") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:32.699177 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:32.699041 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:32.699177 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:32.699106 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs podName:4a509700-2c8d-4113-9121-3ec287223f70 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:36.699090454 +0000 UTC m=+9.372228537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs") pod "network-metrics-daemon-55qcs" (UID: "4a509700-2c8d-4113-9121-3ec287223f70") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:32.799377 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:32.799343 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbx8t\" (UniqueName: \"kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t\") pod \"network-check-target-27hf6\" (UID: \"4a5449c3-4f21-44d2-9d10-9a0956aefdcf\") " pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:32.799594 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:32.799575 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:32.799678 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:32.799606 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:32.799678 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:32.799627 2573 projected.go:196] Error preparing data for projected volume kube-api-access-fbx8t for pod openshift-network-diagnostics/network-check-target-27hf6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:32.799789 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:32.799713 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t podName:4a5449c3-4f21-44d2-9d10-9a0956aefdcf nodeName:}" failed. No retries permitted until 2026-11-04 05:26:36.799693972 +0000 UTC m=+9.472832064 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbx8t" (UniqueName: "kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t") pod "network-check-target-27hf6" (UID: "4a5449c3-4f21-44d2-9d10-9a0956aefdcf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:32.889267 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:32.885948 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:32.890488 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:32.889770 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:32.890488 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:32.889811 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:32.890488 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:32.889974 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55qcs" podUID="4a509700-2c8d-4113-9121-3ec287223f70" Nov 04 05:26:32.890488 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:32.890431 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27hf6" podUID="4a5449c3-4f21-44d2-9d10-9a0956aefdcf" Nov 04 05:26:32.890777 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:32.890549 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nk2gk" podUID="26ae8a36-06bf-414e-b2bd-325033800b72" Nov 04 05:26:34.882716 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:34.881977 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:34.882716 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:34.882252 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nk2gk" podUID="26ae8a36-06bf-414e-b2bd-325033800b72" Nov 04 05:26:34.882716 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:34.882534 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:34.882716 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:34.882596 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27hf6" podUID="4a5449c3-4f21-44d2-9d10-9a0956aefdcf" Nov 04 05:26:34.882716 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:34.882615 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:34.882716 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:34.882686 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55qcs" podUID="4a509700-2c8d-4113-9121-3ec287223f70" Nov 04 05:26:34.926109 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:34.925455 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" event={"ID":"9d7645ed4719bb9894aec65f339b9ca6","Type":"ContainerStarted","Data":"9cd00cc4a3f6d8dd6a55c75aa41137f2b722505b0fe10530c350bde54f4604cc"} Nov 04 05:26:34.929005 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:34.928464 2573 generic.go:356] "Generic (PLEG): container finished" podID="94f347668688382ceddf8a0c03be4355" containerID="0e9b5e18139c670f433c1cf7029708506efb8e28e9f271d6942a649862096747" exitCode=0 Nov 04 05:26:34.929005 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:34.928532 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" event={"ID":"94f347668688382ceddf8a0c03be4355","Type":"ContainerDied","Data":"0e9b5e18139c670f433c1cf7029708506efb8e28e9f271d6942a649862096747"} Nov 04 05:26:34.932693 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:34.932593 2573 generic.go:356] "Generic (PLEG): container finished" podID="ef5a3412-cd67-424f-bb1f-1ec122d5e3eb" containerID="b150a84ce212f377826faaa957ce27d0a1d16422427b3db903dcb1bc94f7ef9e" exitCode=0 Nov 04 05:26:34.932693 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:34.932626 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddzfn" event={"ID":"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb","Type":"ContainerDied","Data":"b150a84ce212f377826faaa957ce27d0a1d16422427b3db903dcb1bc94f7ef9e"} Nov 04 05:26:35.142528 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:35.142061 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" podStartSLOduration=7.142046843 podStartE2EDuration="7.142046843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:35.032718589 +0000 UTC m=+7.705856695" watchObservedRunningTime="2026-11-04 05:26:35.142046843 +0000 UTC m=+7.815184970" Nov 04 05:26:35.949682 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:35.949622 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" event={"ID":"94f347668688382ceddf8a0c03be4355","Type":"ContainerStarted","Data":"d02cc59c406666fc0b1553ca8a0c89808e3a58e8a347fdc7ea7d754928c04d63"} Nov 04 05:26:35.961528 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:35.961504 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x27vz" event={"ID":"1a566754-7023-408d-bb6a-3eb984ffa7e3","Type":"ContainerStarted","Data":"38350f2c1a6e4d118791384c83b4a4fcb54d2f9f838f3504ec0fa6e58b8d81b7"} Nov 04 05:26:36.092218 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:36.092188 2573 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:26:36.092897 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:36.092872 2573 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:26:36.166092 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:36.166015 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" podStartSLOduration=8.165992235 podStartE2EDuration="8.165992235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:36.059288835 +0000 UTC m=+8.732426941" watchObservedRunningTime="2026-11-04 05:26:36.165992235 +0000 UTC m=+8.839130341" Nov 04 05:26:36.274479 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:36.274378 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x27vz" podStartSLOduration=3.134181669 podStartE2EDuration="8.274357972s" totalImagesPullingTime="5.140176303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:36.16673969 +0000 UTC m=+8.839877795" watchObservedRunningTime="2026-11-04 05:26:36.274357972 +0000 UTC m=+8.947496079" Nov 04 05:26:36.737856 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:36.737160 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:36.737856 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:36.737210 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:36.737856 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:36.737338 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:36.737856 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:36.737399 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret podName:26ae8a36-06bf-414e-b2bd-325033800b72 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:44.737378943 +0000 UTC m=+17.410517032 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret") pod "global-pull-secret-syncer-nk2gk" (UID: "26ae8a36-06bf-414e-b2bd-325033800b72") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:36.737856 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:36.737775 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:36.737856 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:36.737826 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs podName:4a509700-2c8d-4113-9121-3ec287223f70 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:44.737811853 +0000 UTC m=+17.410949951 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs") pod "network-metrics-daemon-55qcs" (UID: "4a509700-2c8d-4113-9121-3ec287223f70") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:36.838802 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:36.838464 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbx8t\" (UniqueName: \"kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t\") pod \"network-check-target-27hf6\" (UID: \"4a5449c3-4f21-44d2-9d10-9a0956aefdcf\") " pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:36.838802 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:36.838678 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:36.838802 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:36.838698 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:36.838802 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:36.838711 2573 projected.go:196] Error preparing data for projected volume kube-api-access-fbx8t for pod openshift-network-diagnostics/network-check-target-27hf6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:36.838802 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:36.838764 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t podName:4a5449c3-4f21-44d2-9d10-9a0956aefdcf nodeName:}" failed. No retries permitted until 2026-11-04 05:26:44.838747665 +0000 UTC m=+17.511885761 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbx8t" (UniqueName: "kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t") pod "network-check-target-27hf6" (UID: "4a5449c3-4f21-44d2-9d10-9a0956aefdcf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:36.882224 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:36.882197 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:36.882354 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:36.882331 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55qcs" podUID="4a509700-2c8d-4113-9121-3ec287223f70" Nov 04 05:26:36.882427 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:36.882407 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:36.882507 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:36.882487 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nk2gk" podUID="26ae8a36-06bf-414e-b2bd-325033800b72" Nov 04 05:26:36.882563 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:36.882549 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:36.882631 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:36.882611 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27hf6" podUID="4a5449c3-4f21-44d2-9d10-9a0956aefdcf" Nov 04 05:26:36.972759 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:36.972732 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" event={"ID":"38b751da-304f-4000-bf56-85e1d9aa8152","Type":"ContainerStarted","Data":"909863b4183dce8866c40152b1f331b3329428a157039cf69c322044c2aa3802"} Nov 04 05:26:37.978049 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:37.978021 2573 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 05:26:37.979138 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:37.979023 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m9v8n" event={"ID":"5e7c38cd-3943-484d-b433-cbec7bce35c6","Type":"ContainerStarted","Data":"e55ae39bba070a29a8ed2b0ad70366fe4fce4d023dec625903f63fd92e1ed250"} Nov 04 05:26:38.882202 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:38.882115 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:38.882349 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:38.882260 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55qcs" podUID="4a509700-2c8d-4113-9121-3ec287223f70" Nov 04 05:26:38.882694 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:38.882654 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:38.882791 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:38.882770 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nk2gk" podUID="26ae8a36-06bf-414e-b2bd-325033800b72" Nov 04 05:26:38.882852 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:38.882829 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:38.882922 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:38.882894 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27hf6" podUID="4a5449c3-4f21-44d2-9d10-9a0956aefdcf" Nov 04 05:26:38.983788 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:38.983718 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vmgxh" event={"ID":"77f9e498-8338-4f89-ae13-e72e0108a602","Type":"ContainerStarted","Data":"2663463e8cee6e503321e947ac584b6be6df8cf7c8b3134c52c8013f6541c550"} Nov 04 05:26:39.093390 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:39.093339 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-m9v8n" podStartSLOduration=4.465045617 podStartE2EDuration="11.093327079s" totalImagesPullingTime="6.628281462s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:38.086464989 +0000 UTC m=+10.759603098" watchObservedRunningTime="2026-11-04 05:26:39.093327079 +0000 UTC m=+11.766465184" Nov 04 05:26:39.093604 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:39.093448 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-vmgxh" podStartSLOduration=2.805348309 podStartE2EDuration="11.09344484s" totalImagesPullingTime="8.288096531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:39.093198719 +0000 UTC m=+11.766336825" watchObservedRunningTime="2026-11-04 05:26:39.09344484 +0000 UTC m=+11.766582946" Nov 04 05:26:39.256490 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:39.256239 2573 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 05:26:39.809022 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:39.808901 2573 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T05:26:39.25647639Z","UUID":"1e0d5983-86e0-4428-8e83-c6586bdc7981","Handler":null,"Name":"","Endpoint":""} Nov 04 05:26:39.810784 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:39.810762 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 05:26:39.810911 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:39.810793 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 05:26:39.986860 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:39.986194 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-87f78" event={"ID":"428ba86e-0dd8-4a0a-b899-c7e9d568487c","Type":"ContainerStarted","Data":"7b7f5b9eadfb3e1011616b6fff179044d5b107d95d281ec32360cac343eaa83b"} Nov 04 05:26:39.989928 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:39.989897 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" event={"ID":"38b751da-304f-4000-bf56-85e1d9aa8152","Type":"ContainerStarted","Data":"9916c7c04b9205dcbe055d22c03ef24c63073f908d91a361f7cf838d5361eb0b"} Nov 04 05:26:40.094133 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:40.094079 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-87f78" podStartSLOduration=3.454466937 podStartE2EDuration="12.094060381s" totalImagesPullingTime="8.639593444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:40.09303359 +0000 UTC m=+12.766171695" watchObservedRunningTime="2026-11-04 05:26:40.094060381 +0000 UTC m=+12.767198487" Nov 04 05:26:40.882524 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:40.882494 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:40.882726 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:40.882607 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27hf6" podUID="4a5449c3-4f21-44d2-9d10-9a0956aefdcf" Nov 04 05:26:40.883023 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:40.883001 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:40.883126 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:40.883110 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55qcs" podUID="4a509700-2c8d-4113-9121-3ec287223f70" Nov 04 05:26:40.883182 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:40.883156 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:40.883221 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:40.883207 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nk2gk" podUID="26ae8a36-06bf-414e-b2bd-325033800b72" Nov 04 05:26:40.994242 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:40.994212 2573 generic.go:356] "Generic (PLEG): container finished" podID="ef5a3412-cd67-424f-bb1f-1ec122d5e3eb" containerID="3d7a67791b86e74c89f18c36b21dcb03dd50729fc1c8139b18ea87c1fcb158c6" exitCode=0 Nov 04 05:26:40.994697 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:40.994294 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddzfn" event={"ID":"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb","Type":"ContainerDied","Data":"3d7a67791b86e74c89f18c36b21dcb03dd50729fc1c8139b18ea87c1fcb158c6"} Nov 04 05:26:40.997376 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:40.997298 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q7mtr" event={"ID":"6cf77b4d-32f5-42db-851f-3466ecf583c9","Type":"ContainerStarted","Data":"59fb3729016c496e2b1e9960ae0311833c555ae347195a53abf57f6dcf48b828"} Nov 04 05:26:41.259446 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:41.259398 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-q7mtr" podStartSLOduration=4.879107397 podStartE2EDuration="13.25938425s" totalImagesPullingTime="8.380276853s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:41.25915982 +0000 UTC m=+13.932297928" watchObservedRunningTime="2026-11-04 05:26:41.25938425 +0000 UTC m=+13.932522356" Nov 04 05:26:42.001072 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:42.001032 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" event={"ID":"38b751da-304f-4000-bf56-85e1d9aa8152","Type":"ContainerStarted","Data":"90f6499c41ccb1911d556e8e81a139a726eaac951df4585b638216f40d3d624d"} Nov 04 05:26:42.002722 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:42.002691 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" event={"ID":"8f7d857f-84f9-4bde-9b5a-66164c90377e","Type":"ContainerStarted","Data":"bb8ad3edb543d67519de7a8de58223bf7ccd85bccf47b59a8655f89e91cc1781"} Nov 04 05:26:42.111929 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:42.111884 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" podStartSLOduration=3.885572001 podStartE2EDuration="14.111870772s" totalImagesPullingTime="10.226298771s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:42.111713218 +0000 UTC m=+14.784851324" watchObservedRunningTime="2026-11-04 05:26:42.111870772 +0000 UTC m=+14.785008888" Nov 04 05:26:42.221107 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:42.221056 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" podStartSLOduration=3.373266087 podStartE2EDuration="14.221047359s" totalImagesPullingTime="10.847781272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:42.220985704 +0000 UTC m=+14.894123810" watchObservedRunningTime="2026-11-04 05:26:42.221047359 +0000 UTC m=+14.894185463" Nov 04 05:26:42.881823 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:42.881793 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:42.881823 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:42.881821 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:42.882038 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:42.881832 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:42.882038 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:42.881916 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55qcs" podUID="4a509700-2c8d-4113-9121-3ec287223f70" Nov 04 05:26:42.882117 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:42.882031 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nk2gk" podUID="26ae8a36-06bf-414e-b2bd-325033800b72" Nov 04 05:26:42.882166 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:42.882116 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27hf6" podUID="4a5449c3-4f21-44d2-9d10-9a0956aefdcf" Nov 04 05:26:44.009988 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:44.009797 2573 generic.go:356] "Generic (PLEG): container finished" podID="ef5a3412-cd67-424f-bb1f-1ec122d5e3eb" containerID="237e08585ba6eed1150f0ab2042140d946d4b7d49218a505161a8267b1728265" exitCode=0 Nov 04 05:26:44.010589 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:44.009889 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddzfn" event={"ID":"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb","Type":"ContainerDied","Data":"237e08585ba6eed1150f0ab2042140d946d4b7d49218a505161a8267b1728265"} Nov 04 05:26:44.805878 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:44.805849 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:44.806062 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:44.805891 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:44.806062 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:44.806016 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:44.806062 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:44.806017 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:44.806062 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:44.806064 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret podName:26ae8a36-06bf-414e-b2bd-325033800b72 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:00.806052826 +0000 UTC m=+33.479190910 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret") pod "global-pull-secret-syncer-nk2gk" (UID: "26ae8a36-06bf-414e-b2bd-325033800b72") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:44.806247 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:44.806095 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs podName:4a509700-2c8d-4113-9121-3ec287223f70 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:00.806076572 +0000 UTC m=+33.479214669 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs") pod "network-metrics-daemon-55qcs" (UID: "4a509700-2c8d-4113-9121-3ec287223f70") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:44.882270 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:44.882243 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:44.882430 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:44.882284 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:44.882430 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:44.882363 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55qcs" podUID="4a509700-2c8d-4113-9121-3ec287223f70" Nov 04 05:26:44.882430 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:44.882243 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:44.882545 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:44.882446 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27hf6" podUID="4a5449c3-4f21-44d2-9d10-9a0956aefdcf" Nov 04 05:26:44.882545 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:44.882516 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nk2gk" podUID="26ae8a36-06bf-414e-b2bd-325033800b72" Nov 04 05:26:44.906365 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:44.906340 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbx8t\" (UniqueName: \"kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t\") pod \"network-check-target-27hf6\" (UID: \"4a5449c3-4f21-44d2-9d10-9a0956aefdcf\") " pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:44.906496 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:44.906471 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:44.906496 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:44.906493 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:44.906570 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:44.906506 2573 projected.go:196] Error preparing data for projected volume kube-api-access-fbx8t for pod openshift-network-diagnostics/network-check-target-27hf6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:44.906570 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:44.906553 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t podName:4a5449c3-4f21-44d2-9d10-9a0956aefdcf nodeName:}" failed. No retries permitted until 2026-11-04 05:27:00.906539842 +0000 UTC m=+33.579677946 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbx8t" (UniqueName: "kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t") pod "network-check-target-27hf6" (UID: "4a5449c3-4f21-44d2-9d10-9a0956aefdcf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:45.176422 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:45.176398 2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:26:45.176804 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:45.176519 2573 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 05:26:45.177182 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:45.177162 2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:26:46.019861 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:46.019683 2573 generic.go:356] "Generic (PLEG): container finished" podID="ef5a3412-cd67-424f-bb1f-1ec122d5e3eb" containerID="ee6bdc76eeb9716dc5a327d5d328c4bd6ba78a73189e2c7be25e8fb314ed39bb" exitCode=0 Nov 04 05:26:46.020103 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:46.019766 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddzfn" event={"ID":"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb","Type":"ContainerDied","Data":"ee6bdc76eeb9716dc5a327d5d328c4bd6ba78a73189e2c7be25e8fb314ed39bb"} Nov 04 05:26:46.023284 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:46.023263 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" event={"ID":"72dd8eb9-265d-4761-a507-d91e29606698","Type":"ContainerStarted","Data":"60a9b2f1414506676b08c02d10b97fc73a62bb9ad4e87bc6741cb0b40f3ed57a"} Nov 04 05:26:46.023370 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:46.023320 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" event={"ID":"72dd8eb9-265d-4761-a507-d91e29606698","Type":"ContainerStarted","Data":"f8e00eb0b294876b392a3c265b5bcfe35131fd628a9e7b43951c5a50fb52856a"} Nov 04 05:26:46.023370 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:46.023349 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" event={"ID":"72dd8eb9-265d-4761-a507-d91e29606698","Type":"ContainerStarted","Data":"95696c6b2cc8135b465bfaa7475c871db4899d0459e6e64adac0d615ea21d7e9"} Nov 04 05:26:46.023428 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:46.023382 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" event={"ID":"72dd8eb9-265d-4761-a507-d91e29606698","Type":"ContainerStarted","Data":"5895ecb6328329f2d0a3dc9192468c2c9e009ed49558cdaf31d5900e7a31b1e1"} Nov 04 05:26:46.023428 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:46.023415 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" event={"ID":"72dd8eb9-265d-4761-a507-d91e29606698","Type":"ContainerStarted","Data":"2c8ddf201fbf1d5feeff5c267c543a58293a4f326d0380d37b8d17b9f7c9418c"} Nov 04 05:26:46.023480 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:46.023441 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" event={"ID":"72dd8eb9-265d-4761-a507-d91e29606698","Type":"ContainerStarted","Data":"7e78075a825234b527192e2bf6c5850086966d0f4a71007d76292808d4de7929"} Nov 04 05:26:46.882683 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:46.882639 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:46.883118 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:46.882722 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:46.883118 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:46.882740 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:46.883118 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:46.882820 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nk2gk" podUID="26ae8a36-06bf-414e-b2bd-325033800b72" Nov 04 05:26:46.883118 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:46.882933 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55qcs" podUID="4a509700-2c8d-4113-9121-3ec287223f70" Nov 04 05:26:46.883118 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:46.883010 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27hf6" podUID="4a5449c3-4f21-44d2-9d10-9a0956aefdcf" Nov 04 05:26:48.031395 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:48.031318 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" event={"ID":"72dd8eb9-265d-4761-a507-d91e29606698","Type":"ContainerStarted","Data":"b917ebc8b1a071d7f4330b2a69535b02fdfec7b69dd5d23b17015882aa802ff1"} Nov 04 05:26:48.882236 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:48.882211 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:48.882387 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:48.882286 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:48.882387 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:48.882310 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:48.882497 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:48.882381 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27hf6" podUID="4a5449c3-4f21-44d2-9d10-9a0956aefdcf" Nov 04 05:26:48.882545 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:48.882481 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55qcs" podUID="4a509700-2c8d-4113-9121-3ec287223f70" Nov 04 05:26:48.882580 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:48.882550 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nk2gk" podUID="26ae8a36-06bf-414e-b2bd-325033800b72" Nov 04 05:26:49.035305 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:49.035283 2573 generic.go:356] "Generic (PLEG): container finished" podID="ef5a3412-cd67-424f-bb1f-1ec122d5e3eb" containerID="3083708d8f0d881f974705f48071193cc43c964153052ec1212abf95cc0d8a4e" exitCode=0 Nov 04 05:26:49.035686 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:49.035333 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddzfn" event={"ID":"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb","Type":"ContainerDied","Data":"3083708d8f0d881f974705f48071193cc43c964153052ec1212abf95cc0d8a4e"} Nov 04 05:26:50.040518 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:50.040326 2573 generic.go:356] "Generic (PLEG): container finished" podID="ef5a3412-cd67-424f-bb1f-1ec122d5e3eb" containerID="f97fd050565f600b43a37a265bce54feb2e241b2f75663ba5c6b30ece348fce1" exitCode=0 Nov 04 05:26:50.041201 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:50.040401 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddzfn" event={"ID":"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb","Type":"ContainerDied","Data":"f97fd050565f600b43a37a265bce54feb2e241b2f75663ba5c6b30ece348fce1"} Nov 04 05:26:50.044222 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:50.044198 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" event={"ID":"72dd8eb9-265d-4761-a507-d91e29606698","Type":"ContainerStarted","Data":"8314698b7150025f80f58e98b560b1af3af91603ea89c38460570ca44982debd"} Nov 04 05:26:50.044416 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:50.044398 2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:50.044492 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:50.044473 2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:50.060918 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:50.060836 2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:50.062314 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:50.062296 2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:50.351132 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:50.351052 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" podStartSLOduration=7.537798817 podStartE2EDuration="22.351039646s" totalImagesPullingTime="14.813240829s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:50.35069165 +0000 UTC m=+23.023829756" watchObservedRunningTime="2026-11-04 05:26:50.351039646 +0000 UTC m=+23.024177751" Nov 04 05:26:50.882454 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:50.882420 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:50.882454 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:50.882452 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:50.882454 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:50.882456 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:50.882706 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:50.882526 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27hf6" podUID="4a5449c3-4f21-44d2-9d10-9a0956aefdcf" Nov 04 05:26:50.882706 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:50.882596 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nk2gk" podUID="26ae8a36-06bf-414e-b2bd-325033800b72" Nov 04 05:26:50.882706 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:50.882693 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55qcs" podUID="4a509700-2c8d-4113-9121-3ec287223f70" Nov 04 05:26:51.051212 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:51.051176 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddzfn" event={"ID":"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb","Type":"ContainerStarted","Data":"7df58aa99986e7deee3ab7f9957721d36bf1ad1f0089a7af86ecb9eb97da7b80"} Nov 04 05:26:51.051710 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:51.051241 2573 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 05:26:51.164977 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:51.164932 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ddzfn" podStartSLOduration=8.917706508 podStartE2EDuration="23.164919495s" totalImagesPullingTime="14.247212987s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:51.164358775 +0000 UTC m=+23.837496881" watchObservedRunningTime="2026-11-04 05:26:51.164919495 +0000 UTC m=+23.838057600" Nov 04 05:26:52.054068 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:52.054038 2573 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 05:26:52.551987 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:52.551944 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nk2gk"] Nov 04 05:26:52.552142 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:52.552077 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:52.552212 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:52.552188 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nk2gk" podUID="26ae8a36-06bf-414e-b2bd-325033800b72" Nov 04 05:26:52.552745 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:52.552715 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-27hf6"] Nov 04 05:26:52.552873 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:52.552820 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:52.552921 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:52.552901 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27hf6" podUID="4a5449c3-4f21-44d2-9d10-9a0956aefdcf" Nov 04 05:26:52.597684 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:52.597640 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-55qcs"] Nov 04 05:26:52.597833 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:52.597756 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:52.597879 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:52.597834 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55qcs" podUID="4a509700-2c8d-4113-9121-3ec287223f70" Nov 04 05:26:53.882333 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:53.882301 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:53.882754 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:53.882462 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nk2gk" podUID="26ae8a36-06bf-414e-b2bd-325033800b72" Nov 04 05:26:53.882754 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:53.882500 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:53.882754 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:53.882580 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55qcs" podUID="4a509700-2c8d-4113-9121-3ec287223f70" Nov 04 05:26:54.882139 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:54.882070 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:54.882270 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:54.882165 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27hf6" podUID="4a5449c3-4f21-44d2-9d10-9a0956aefdcf" Nov 04 05:26:55.882279 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:55.882250 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:55.882643 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:55.882357 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nk2gk" podUID="26ae8a36-06bf-414e-b2bd-325033800b72" Nov 04 05:26:55.882643 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:55.882434 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:55.882643 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:55.882521 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55qcs" podUID="4a509700-2c8d-4113-9121-3ec287223f70" Nov 04 05:26:56.882615 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:56.882450 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:56.883075 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:56.882723 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27hf6" podUID="4a5449c3-4f21-44d2-9d10-9a0956aefdcf" Nov 04 05:26:57.008861 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:57.008705 2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:57.009089 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:57.009075 2573 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 05:26:57.024045 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:57.024024 2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:26:57.883713 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:57.883680 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:57.884072 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:57.883722 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:57.884072 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:57.883804 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55qcs" podUID="4a509700-2c8d-4113-9121-3ec287223f70" Nov 04 05:26:57.884072 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:57.883913 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nk2gk" podUID="26ae8a36-06bf-414e-b2bd-325033800b72" Nov 04 05:26:58.589125 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.589095 2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeReady" Nov 04 05:26:58.589279 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.589194 2573 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Nov 04 05:26:58.736915 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.736880 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-pw5zp"] Nov 04 05:26:58.756469 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.756445 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-5zj4b"] Nov 04 05:26:58.756612 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.756567 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-pw5zp" Nov 04 05:26:58.768369 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.768350 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-h5ccq"] Nov 04 05:26:58.768478 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.768463 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-5zj4b" Nov 04 05:26:58.782466 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.782442 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6469786477-m4mgb"] Nov 04 05:26:58.782577 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.782559 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-h5ccq" Nov 04 05:26:58.798533 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.798514 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-pgdqn"] Nov 04 05:26:58.798657 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.798640 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:58.808598 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.808577 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9"] Nov 04 05:26:58.810291 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.810258 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Nov 04 05:26:58.811766 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.811746 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:26:58.828456 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.828438 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Nov 04 05:26:58.830766 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.830714 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl"] Nov 04 05:26:58.830854 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.830813 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:26:58.847381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.847334 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Nov 04 05:26:58.847381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.847357 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Nov 04 05:26:58.847560 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.847549 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Nov 04 05:26:58.852914 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.852899 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl"] Nov 04 05:26:58.853011 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.852998 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:26:58.867113 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.867093 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg"] Nov 04 05:26:58.867248 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.867213 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:26:58.883338 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.883320 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8f7d8c7c6-q48ph"] Nov 04 05:26:58.883450 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.883434 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:26:58.901153 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901131 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fbtn\" (UniqueName: \"kubernetes.io/projected/e10001da-2e9e-4b80-b6e0-0f1fca3df854-kube-api-access-2fbtn\") pod \"kas-connection-checker-685895c7c9-h5ccq\" (UID: \"e10001da-2e9e-4b80-b6e0-0f1fca3df854\") " pod="kube-system/kas-connection-checker-685895c7c9-h5ccq" Nov 04 05:26:58.901485 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901163 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6gpf\" (UniqueName: \"kubernetes.io/projected/2e2ba652-6141-4984-8384-51f9720a02ef-kube-api-access-m6gpf\") pod \"kas-connection-checker-685895c7c9-pw5zp\" (UID: \"2e2ba652-6141-4984-8384-51f9720a02ef\") " pod="kube-system/kas-connection-checker-685895c7c9-pw5zp" Nov 04 05:26:58.901485 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901181 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-image-registry-private-configuration\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:58.901485 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901198 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:58.901485 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901257 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-bound-sa-token\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:58.901485 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901336 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws4hd\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-kube-api-access-ws4hd\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:58.901485 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901356 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d26b56ed-c6e3-4321-8044-d29e5fe081c8-config\") pod \"service-ca-operator-544d749c8c-kqsd9\" (UID: \"d26b56ed-c6e3-4321-8044-d29e5fe081c8\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:26:58.901485 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901446 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-certificates\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:58.901485 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901468 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-trusted-ca\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:58.901781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901491 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-installation-pull-secrets\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:58.901781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901506 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2flm\" (UniqueName: \"kubernetes.io/projected/523aff5c-ac2b-43be-8e13-792f44b2c14e-kube-api-access-c2flm\") pod \"kas-connection-checker-685895c7c9-5zj4b\" (UID: \"523aff5c-ac2b-43be-8e13-792f44b2c14e\") " pod="kube-system/kas-connection-checker-685895c7c9-5zj4b" Nov 04 05:26:58.901781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901528 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/00a06b1e-791a-4652-823c-b73672b7e4b2-ca-trust-extracted\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:58.901781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901569 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-58c967f969-4xztc"] Nov 04 05:26:58.901781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901642 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a35474ea-79b9-4d84-81b1-39db689685d2-nginx-conf\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:26:58.901781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901651 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:58.901781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901705 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:26:58.901781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901738 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwj9c\" (UniqueName: \"kubernetes.io/projected/d26b56ed-c6e3-4321-8044-d29e5fe081c8-kube-api-access-xwj9c\") pod \"service-ca-operator-544d749c8c-kqsd9\" (UID: \"d26b56ed-c6e3-4321-8044-d29e5fe081c8\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:26:58.901781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901763 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:26:58.901781 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.901775 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d26b56ed-c6e3-4321-8044-d29e5fe081c8-serving-cert\") pod \"service-ca-operator-544d749c8c-kqsd9\" (UID: \"d26b56ed-c6e3-4321-8044-d29e5fe081c8\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:26:58.910143 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.910124 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Nov 04 05:26:58.910143 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.910140 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:26:58.910280 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.910128 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:26:58.910280 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.910124 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Nov 04 05:26:58.910280 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.910181 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Nov 04 05:26:58.910280 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.910189 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Nov 04 05:26:58.910280 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.910229 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Nov 04 05:26:58.911507 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.911490 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Nov 04 05:26:58.915395 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.915379 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:26:58.915541 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.915526 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Nov 04 05:26:58.915701 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.915688 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-qnfh8"] Nov 04 05:26:58.915752 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.915716 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Nov 04 05:26:58.915815 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.915799 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:26:58.929914 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.929895 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-b24ft"] Nov 04 05:26:58.930020 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.930007 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:26:58.931653 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.931636 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 05:26:58.931723 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.931636 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 05:26:58.932163 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.932150 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Nov 04 05:26:58.942773 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.942759 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j8lxw"] Nov 04 05:26:58.942872 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.942857 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:58.947650 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.947633 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Nov 04 05:26:58.947755 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.947720 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Nov 04 05:26:58.947821 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.947769 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Nov 04 05:26:58.947821 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.947786 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Nov 04 05:26:58.947821 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.947813 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Nov 04 05:26:58.947976 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.947839 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-8r47x\"" Nov 04 05:26:58.949748 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.949722 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 05:26:58.949848 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.949817 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-k2gkz\"" Nov 04 05:26:58.949848 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.949808 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 05:26:58.956199 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.956184 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6qrg6\"" Nov 04 05:26:58.963976 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.963960 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Nov 04 05:26:58.964042 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.963999 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Nov 04 05:26:58.964090 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.964050 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Nov 04 05:26:58.964090 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.964066 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Nov 04 05:26:58.964164 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.964104 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Nov 04 05:26:58.964164 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.964122 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Nov 04 05:26:58.966922 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.966907 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-xk9lw"] Nov 04 05:26:58.967016 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.967003 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j8lxw" Nov 04 05:26:58.988388 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.988373 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-767sh"] Nov 04 05:26:58.988461 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:58.988455 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xk9lw" Nov 04 05:26:59.002318 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.002298 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-29hbl\" (UID: \"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:26:59.002411 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.002328 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-config\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:26:59.002411 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.002351 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8prxh\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-kube-api-access-8prxh\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.002518 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.002411 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:26:59.002518 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.002448 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtv2t\" (UniqueName: \"kubernetes.io/projected/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-kube-api-access-dtv2t\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-29hbl\" (UID: \"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:26:59.002518 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.002488 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fbtn\" (UniqueName: \"kubernetes.io/projected/e10001da-2e9e-4b80-b6e0-0f1fca3df854-kube-api-access-2fbtn\") pod \"kas-connection-checker-685895c7c9-h5ccq\" (UID: \"e10001da-2e9e-4b80-b6e0-0f1fca3df854\") " pod="kube-system/kas-connection-checker-685895c7c9-h5ccq" Nov 04 05:26:59.002518 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.002514 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-image-registry-private-configuration\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:59.002701 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.002530 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Nov 04 05:26:59.002701 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.002541 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-29hbl\" (UID: \"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:26:59.002701 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.002567 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:26:59.002701 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.002581 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert podName:a35474ea-79b9-4d84-81b1-39db689685d2 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:59.502564016 +0000 UTC m=+32.175702103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pgdqn" (UID: "a35474ea-79b9-4d84-81b1-39db689685d2") : secret "networking-console-plugin-cert" not found Nov 04 05:26:59.002701 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.002687 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-ca-trust-extracted\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.002942 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.002716 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-bound-sa-token\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.002942 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.002771 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-installation-pull-secrets\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.002942 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.002853 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2flm\" (UniqueName: \"kubernetes.io/projected/523aff5c-ac2b-43be-8e13-792f44b2c14e-kube-api-access-c2flm\") pod \"kas-connection-checker-685895c7c9-5zj4b\" (UID: \"523aff5c-ac2b-43be-8e13-792f44b2c14e\") " pod="kube-system/kas-connection-checker-685895c7c9-5zj4b" Nov 04 05:26:59.002942 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.002881 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-certificates\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.002942 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.002912 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-trusted-ca\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:59.002942 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.002937 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.003245 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.003017 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-stats-auth\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:26:59.005596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.003437 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:26:59.005596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.003572 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/00a06b1e-791a-4652-823c-b73672b7e4b2-ca-trust-extracted\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:59.005596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.003613 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p6g8\" (UniqueName: \"kubernetes.io/projected/d998c080-e5b9-4de6-af9c-68a8c531257c-kube-api-access-5p6g8\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:26:59.005596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.003650 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a35474ea-79b9-4d84-81b1-39db689685d2-nginx-conf\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:26:59.005596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.003707 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-default-certificate\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:26:59.005596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.003737 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwj9c\" (UniqueName: \"kubernetes.io/projected/d26b56ed-c6e3-4321-8044-d29e5fe081c8-kube-api-access-xwj9c\") pod \"service-ca-operator-544d749c8c-kqsd9\" (UID: \"d26b56ed-c6e3-4321-8044-d29e5fe081c8\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:26:59.005596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.003771 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d26b56ed-c6e3-4321-8044-d29e5fe081c8-serving-cert\") pod \"service-ca-operator-544d749c8c-kqsd9\" (UID: \"d26b56ed-c6e3-4321-8044-d29e5fe081c8\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:26:59.005596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.003808 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6gpf\" (UniqueName: \"kubernetes.io/projected/2e2ba652-6141-4984-8384-51f9720a02ef-kube-api-access-m6gpf\") pod \"kas-connection-checker-685895c7c9-pw5zp\" (UID: \"2e2ba652-6141-4984-8384-51f9720a02ef\") " pod="kube-system/kas-connection-checker-685895c7c9-pw5zp" Nov 04 05:26:59.005596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.003839 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws4hd\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-kube-api-access-ws4hd\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:59.005596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.003871 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:59.005596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.003901 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjngp\" (UniqueName: \"kubernetes.io/projected/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-kube-api-access-xjngp\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:26:59.005596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.003932 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-bound-sa-token\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:59.005596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.003964 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:26:59.005596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.003998 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d26b56ed-c6e3-4321-8044-d29e5fe081c8-config\") pod \"service-ca-operator-544d749c8c-kqsd9\" (UID: \"d26b56ed-c6e3-4321-8044-d29e5fe081c8\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:26:59.005596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.004023 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/191dfb71-401c-494a-b7bb-cbc6c3273430-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:26:59.006306 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.004077 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-serving-cert\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:26:59.006306 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.004128 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-certificates\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:59.006306 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.004160 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-installation-pull-secrets\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:59.006306 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.004167 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-trusted-ca\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:59.006306 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.004195 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:26:59.006306 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.004261 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfptj\" (UniqueName: \"kubernetes.io/projected/191dfb71-401c-494a-b7bb-cbc6c3273430-kube-api-access-gfptj\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:26:59.006306 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.004727 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:26:59.006306 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.004742 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6469786477-m4mgb: secret "image-registry-tls" not found Nov 04 05:26:59.006306 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.004793 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls podName:00a06b1e-791a-4652-823c-b73672b7e4b2 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:59.504770493 +0000 UTC m=+32.177908590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls") pod "image-registry-6469786477-m4mgb" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2") : secret "image-registry-tls" not found Nov 04 05:26:59.006306 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.005050 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/00a06b1e-791a-4652-823c-b73672b7e4b2-ca-trust-extracted\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:59.006306 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.005101 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-trusted-ca\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.006306 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.005139 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:26:59.006306 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.005172 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz2hq\" (UniqueName: \"kubernetes.io/projected/c87f7dad-1181-4db5-b846-05d8fb0a33d5-kube-api-access-tz2hq\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:26:59.006306 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.005204 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-trusted-ca\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:26:59.006306 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.005236 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-image-registry-private-configuration\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.009827 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.009807 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-brcbz"] Nov 04 05:26:59.010201 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.010178 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-image-registry-private-configuration\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:59.010329 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.010216 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-767sh" Nov 04 05:26:59.010442 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.010420 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-certificates\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:59.010520 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.010224 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d26b56ed-c6e3-4321-8044-d29e5fe081c8-config\") pod \"service-ca-operator-544d749c8c-kqsd9\" (UID: \"d26b56ed-c6e3-4321-8044-d29e5fe081c8\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:26:59.010577 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.010543 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 05:26:59.010636 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.010614 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Nov 04 05:26:59.010705 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.010653 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Nov 04 05:26:59.010705 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.010651 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a35474ea-79b9-4d84-81b1-39db689685d2-nginx-conf\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:26:59.010705 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.010695 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 05:26:59.010819 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.010217 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-installation-pull-secrets\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:59.010903 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.010887 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:26:59.010975 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.010922 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Nov 04 05:26:59.011931 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.011911 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Nov 04 05:26:59.012393 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.012378 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Nov 04 05:26:59.013049 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.013034 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d26b56ed-c6e3-4321-8044-d29e5fe081c8-serving-cert\") pod \"service-ca-operator-544d749c8c-kqsd9\" (UID: \"d26b56ed-c6e3-4321-8044-d29e5fe081c8\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:26:59.015161 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.015145 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Nov 04 05:26:59.015238 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.015168 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:26:59.033619 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033603 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-pw5zp"] Nov 04 05:26:59.033707 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033630 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-5zj4b"] Nov 04 05:26:59.033707 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033646 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-h5ccq"] Nov 04 05:26:59.033707 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033659 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6469786477-m4mgb"] Nov 04 05:26:59.033707 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033686 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9"] Nov 04 05:26:59.033707 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033700 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl"] Nov 04 05:26:59.033917 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033715 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-pgdqn"] Nov 04 05:26:59.033917 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033717 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-brcbz" Nov 04 05:26:59.033917 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033727 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-qnfh8"] Nov 04 05:26:59.033917 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033740 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-b24ft"] Nov 04 05:26:59.033917 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033754 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-brcbz"] Nov 04 05:26:59.033917 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033770 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg"] Nov 04 05:26:59.033917 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033782 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58c967f969-4xztc"] Nov 04 05:26:59.033917 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033795 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8f7d8c7c6-q48ph"] Nov 04 05:26:59.033917 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033807 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j8lxw"] Nov 04 05:26:59.033917 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033819 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl"] Nov 04 05:26:59.033917 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033835 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-767sh"] Nov 04 05:26:59.033917 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.033848 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-xk9lw"] Nov 04 05:26:59.047894 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.047876 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Nov 04 05:26:59.047965 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.047900 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Nov 04 05:26:59.059300 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.059283 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fbtn\" (UniqueName: \"kubernetes.io/projected/e10001da-2e9e-4b80-b6e0-0f1fca3df854-kube-api-access-2fbtn\") pod \"kas-connection-checker-685895c7c9-h5ccq\" (UID: \"e10001da-2e9e-4b80-b6e0-0f1fca3df854\") " pod="kube-system/kas-connection-checker-685895c7c9-h5ccq" Nov 04 05:26:59.059709 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.059653 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2flm\" (UniqueName: \"kubernetes.io/projected/523aff5c-ac2b-43be-8e13-792f44b2c14e-kube-api-access-c2flm\") pod \"kas-connection-checker-685895c7c9-5zj4b\" (UID: \"523aff5c-ac2b-43be-8e13-792f44b2c14e\") " pod="kube-system/kas-connection-checker-685895c7c9-5zj4b" Nov 04 05:26:59.060393 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.060368 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-bound-sa-token\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:59.061997 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.061961 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws4hd\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-kube-api-access-ws4hd\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:59.062074 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.062000 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6gpf\" (UniqueName: \"kubernetes.io/projected/2e2ba652-6141-4984-8384-51f9720a02ef-kube-api-access-m6gpf\") pod \"kas-connection-checker-685895c7c9-pw5zp\" (UID: \"2e2ba652-6141-4984-8384-51f9720a02ef\") " pod="kube-system/kas-connection-checker-685895c7c9-pw5zp" Nov 04 05:26:59.064098 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.064081 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwj9c\" (UniqueName: \"kubernetes.io/projected/d26b56ed-c6e3-4321-8044-d29e5fe081c8-kube-api-access-xwj9c\") pod \"service-ca-operator-544d749c8c-kqsd9\" (UID: \"d26b56ed-c6e3-4321-8044-d29e5fe081c8\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:26:59.076612 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.076593 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-pw5zp" Nov 04 05:26:59.087573 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.087553 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-5zj4b" Nov 04 05:26:59.105798 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.105778 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-h5ccq" Nov 04 05:26:59.105899 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.105884 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-29hbl\" (UID: \"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:26:59.105943 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.105908 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-config\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:26:59.105943 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.105924 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8prxh\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-kube-api-access-8prxh\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.106012 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.105958 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtv2t\" (UniqueName: \"kubernetes.io/projected/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-kube-api-access-dtv2t\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-29hbl\" (UID: \"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:26:59.106012 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.105986 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/886edd3a-6e51-4ccf-92a8-9d8a3d447449-config-volume\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:26:59.106119 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106017 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-29hbl\" (UID: \"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:26:59.106119 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106042 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:26:59.106215 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.106158 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Nov 04 05:26:59.106265 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.106216 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls podName:191dfb71-401c-494a-b7bb-cbc6c3273430 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:59.606199151 +0000 UTC m=+32.279337253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-hcsmg" (UID: "191dfb71-401c-494a-b7bb-cbc6c3273430") : secret "cluster-monitoring-operator-tls" not found Nov 04 05:26:59.106265 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106160 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-ca-trust-extracted\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.106368 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106296 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-bound-sa-token\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.106368 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106325 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-installation-pull-secrets\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.106459 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106373 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-certificates\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.106459 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106400 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-29hbl\" (UID: \"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:26:59.106459 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106410 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqqt4\" (UniqueName: \"kubernetes.io/projected/5704adc4-5d8d-439e-ae45-2ab73b06bf14-kube-api-access-vqqt4\") pod \"ingress-canary-767sh\" (UID: \"5704adc4-5d8d-439e-ae45-2ab73b06bf14\") " pod="openshift-ingress-canary/ingress-canary-767sh" Nov 04 05:26:59.106596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106462 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-ca-trust-extracted\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.106596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106504 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.106596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106533 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-stats-auth\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:26:59.106596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106556 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:26:59.106596 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106580 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-config\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:26:59.106596 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.106588 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:26:59.106902 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.106602 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8f7d8c7c6-q48ph: secret "image-registry-tls" not found Nov 04 05:26:59.106902 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106580 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90fb287e-4af9-4e0b-919d-00713b37605c-tmp\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.106902 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106646 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/886edd3a-6e51-4ccf-92a8-9d8a3d447449-tmp-dir\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:26:59.106902 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.106701 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls podName:aa931f6e-a107-4ca3-bb54-e89a91d0f67a nodeName:}" failed. No retries permitted until 2026-11-04 05:26:59.606682935 +0000 UTC m=+32.279821035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls") pod "image-registry-8f7d8c7c6-q48ph" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a") : secret "image-registry-tls" not found Nov 04 05:26:59.106902 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.106702 2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Nov 04 05:26:59.106902 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.106756 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:59.606742058 +0000 UTC m=+32.279880149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : secret "router-metrics-certs-default" not found Nov 04 05:26:59.106902 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106775 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90fb287e-4af9-4e0b-919d-00713b37605c-service-ca-bundle\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.106902 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106807 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90fb287e-4af9-4e0b-919d-00713b37605c-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.106902 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106830 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7488\" (UniqueName: \"kubernetes.io/projected/90fb287e-4af9-4e0b-919d-00713b37605c-kube-api-access-m7488\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.106902 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106855 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/90fb287e-4af9-4e0b-919d-00713b37605c-serving-cert\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.107318 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.106920 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p6g8\" (UniqueName: \"kubernetes.io/projected/d998c080-e5b9-4de6-af9c-68a8c531257c-kube-api-access-5p6g8\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:26:59.107318 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107001 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-default-certificate\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:26:59.107318 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107024 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-certificates\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.107318 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107033 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:26:59.107318 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107081 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/90fb287e-4af9-4e0b-919d-00713b37605c-snapshots\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.107318 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107134 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjngp\" (UniqueName: \"kubernetes.io/projected/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-kube-api-access-xjngp\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:26:59.107318 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107180 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:26:59.107318 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107257 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7x5w\" (UniqueName: \"kubernetes.io/projected/a10e4998-0b1d-4041-a022-9e36300936fa-kube-api-access-n7x5w\") pod \"volume-data-source-validator-58c9f5f46d-j8lxw\" (UID: \"a10e4998-0b1d-4041-a022-9e36300936fa\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j8lxw" Nov 04 05:26:59.107318 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.107274 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:59.607258243 +0000 UTC m=+32.280396345 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : configmap references non-existent config key: service-ca.crt Nov 04 05:26:59.107755 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107461 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/191dfb71-401c-494a-b7bb-cbc6c3273430-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:26:59.107755 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107490 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbg2j\" (UniqueName: \"kubernetes.io/projected/886edd3a-6e51-4ccf-92a8-9d8a3d447449-kube-api-access-hbg2j\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:26:59.107755 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107506 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-serving-cert\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:26:59.107755 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107540 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:26:59.107755 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107572 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rvs5\" (UniqueName: \"kubernetes.io/projected/b25ddbfa-3e3e-49fd-870f-963d087f22a5-kube-api-access-5rvs5\") pod \"network-check-source-6f78d59f99-xk9lw\" (UID: \"b25ddbfa-3e3e-49fd-870f-963d087f22a5\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xk9lw" Nov 04 05:26:59.107755 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107614 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfptj\" (UniqueName: \"kubernetes.io/projected/191dfb71-401c-494a-b7bb-cbc6c3273430-kube-api-access-gfptj\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:26:59.107755 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107640 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-trusted-ca\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.107755 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107685 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:26:59.107755 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107730 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz2hq\" (UniqueName: \"kubernetes.io/projected/c87f7dad-1181-4db5-b846-05d8fb0a33d5-kube-api-access-tz2hq\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:26:59.109161 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107783 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-trusted-ca\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:26:59.109161 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107810 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-image-registry-private-configuration\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.109161 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.107838 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5704adc4-5d8d-439e-ae45-2ab73b06bf14-cert\") pod \"ingress-canary-767sh\" (UID: \"5704adc4-5d8d-439e-ae45-2ab73b06bf14\") " pod="openshift-ingress-canary/ingress-canary-767sh" Nov 04 05:26:59.109161 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.108124 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Nov 04 05:26:59.109161 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.108172 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls podName:d998c080-e5b9-4de6-af9c-68a8c531257c nodeName:}" failed. No retries permitted until 2026-11-04 05:26:59.608158006 +0000 UTC m=+32.281296105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6nlwl" (UID: "d998c080-e5b9-4de6-af9c-68a8c531257c") : secret "samples-operator-tls" not found Nov 04 05:26:59.109161 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.108196 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:26:59.109161 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.108219 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/191dfb71-401c-494a-b7bb-cbc6c3273430-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:26:59.109161 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.108645 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-trusted-ca\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.109161 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.108857 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-installation-pull-secrets\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.109161 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.108940 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-trusted-ca\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:26:59.109648 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.109306 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-29hbl\" (UID: \"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:26:59.109648 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.109305 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-stats-auth\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:26:59.110211 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.110189 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-image-registry-private-configuration\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.110379 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.110352 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-serving-cert\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:26:59.110471 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.110454 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-default-certificate\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:26:59.110520 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.110467 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 05:26:59.110520 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.110506 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 05:26:59.110520 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.110511 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 05:26:59.148315 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.148282 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tb5s5\"" Nov 04 05:26:59.148404 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.148380 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 05:26:59.148451 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.148437 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 05:26:59.148535 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.148524 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-bnzsz\"" Nov 04 05:26:59.164304 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.164124 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:26:59.170196 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.170167 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz2hq\" (UniqueName: \"kubernetes.io/projected/c87f7dad-1181-4db5-b846-05d8fb0a33d5-kube-api-access-tz2hq\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:26:59.170393 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.170283 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjngp\" (UniqueName: \"kubernetes.io/projected/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-kube-api-access-xjngp\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:26:59.172310 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.172253 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfptj\" (UniqueName: \"kubernetes.io/projected/191dfb71-401c-494a-b7bb-cbc6c3273430-kube-api-access-gfptj\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:26:59.195376 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.181218 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-bound-sa-token\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.195376 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.181232 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtv2t\" (UniqueName: \"kubernetes.io/projected/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-kube-api-access-dtv2t\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-29hbl\" (UID: \"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:26:59.195376 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.184822 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:26:59.195376 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.192373 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p6g8\" (UniqueName: \"kubernetes.io/projected/d998c080-e5b9-4de6-af9c-68a8c531257c-kube-api-access-5p6g8\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:26:59.195376 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.194454 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8prxh\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-kube-api-access-8prxh\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.209183 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.209145 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqqt4\" (UniqueName: \"kubernetes.io/projected/5704adc4-5d8d-439e-ae45-2ab73b06bf14-kube-api-access-vqqt4\") pod \"ingress-canary-767sh\" (UID: \"5704adc4-5d8d-439e-ae45-2ab73b06bf14\") " pod="openshift-ingress-canary/ingress-canary-767sh" Nov 04 05:26:59.209293 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.209216 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90fb287e-4af9-4e0b-919d-00713b37605c-tmp\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.209293 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.209248 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/886edd3a-6e51-4ccf-92a8-9d8a3d447449-tmp-dir\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:26:59.209293 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.209278 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90fb287e-4af9-4e0b-919d-00713b37605c-service-ca-bundle\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.209478 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.209306 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90fb287e-4af9-4e0b-919d-00713b37605c-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.209478 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.209332 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7488\" (UniqueName: \"kubernetes.io/projected/90fb287e-4af9-4e0b-919d-00713b37605c-kube-api-access-m7488\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.209478 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.209357 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/90fb287e-4af9-4e0b-919d-00713b37605c-serving-cert\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.209478 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.209389 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:26:59.209478 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.209422 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/90fb287e-4af9-4e0b-919d-00713b37605c-snapshots\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.210130 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.210109 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90fb287e-4af9-4e0b-919d-00713b37605c-service-ca-bundle\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.210343 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.210317 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7x5w\" (UniqueName: \"kubernetes.io/projected/a10e4998-0b1d-4041-a022-9e36300936fa-kube-api-access-n7x5w\") pod \"volume-data-source-validator-58c9f5f46d-j8lxw\" (UID: \"a10e4998-0b1d-4041-a022-9e36300936fa\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j8lxw" Nov 04 05:26:59.210428 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.210376 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbg2j\" (UniqueName: \"kubernetes.io/projected/886edd3a-6e51-4ccf-92a8-9d8a3d447449-kube-api-access-hbg2j\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:26:59.210428 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.210419 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rvs5\" (UniqueName: \"kubernetes.io/projected/b25ddbfa-3e3e-49fd-870f-963d087f22a5-kube-api-access-5rvs5\") pod \"network-check-source-6f78d59f99-xk9lw\" (UID: \"b25ddbfa-3e3e-49fd-870f-963d087f22a5\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xk9lw" Nov 04 05:26:59.210524 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.210471 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5704adc4-5d8d-439e-ae45-2ab73b06bf14-cert\") pod \"ingress-canary-767sh\" (UID: \"5704adc4-5d8d-439e-ae45-2ab73b06bf14\") " pod="openshift-ingress-canary/ingress-canary-767sh" Nov 04 05:26:59.210571 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.210522 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/886edd3a-6e51-4ccf-92a8-9d8a3d447449-config-volume\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:26:59.210571 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.210554 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90fb287e-4af9-4e0b-919d-00713b37605c-tmp\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.211484 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.210688 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90fb287e-4af9-4e0b-919d-00713b37605c-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.211484 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.211030 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/90fb287e-4af9-4e0b-919d-00713b37605c-snapshots\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.211484 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.211097 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/886edd3a-6e51-4ccf-92a8-9d8a3d447449-config-volume\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:26:59.211484 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.211153 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:26:59.211484 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.211206 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls podName:886edd3a-6e51-4ccf-92a8-9d8a3d447449 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:59.711189117 +0000 UTC m=+32.384327207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls") pod "dns-default-brcbz" (UID: "886edd3a-6e51-4ccf-92a8-9d8a3d447449") : secret "dns-default-metrics-tls" not found Nov 04 05:26:59.211484 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.211423 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/886edd3a-6e51-4ccf-92a8-9d8a3d447449-tmp-dir\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:26:59.214512 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.214458 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/90fb287e-4af9-4e0b-919d-00713b37605c-serving-cert\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.216081 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.216060 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5704adc4-5d8d-439e-ae45-2ab73b06bf14-cert\") pod \"ingress-canary-767sh\" (UID: \"5704adc4-5d8d-439e-ae45-2ab73b06bf14\") " pod="openshift-ingress-canary/ingress-canary-767sh" Nov 04 05:26:59.258835 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.258481 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:26:59.269846 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.269780 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7488\" (UniqueName: \"kubernetes.io/projected/90fb287e-4af9-4e0b-919d-00713b37605c-kube-api-access-m7488\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.270826 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.270806 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rvs5\" (UniqueName: \"kubernetes.io/projected/b25ddbfa-3e3e-49fd-870f-963d087f22a5-kube-api-access-5rvs5\") pod \"network-check-source-6f78d59f99-xk9lw\" (UID: \"b25ddbfa-3e3e-49fd-870f-963d087f22a5\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xk9lw" Nov 04 05:26:59.271508 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.271084 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbg2j\" (UniqueName: \"kubernetes.io/projected/886edd3a-6e51-4ccf-92a8-9d8a3d447449-kube-api-access-hbg2j\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:26:59.271508 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.271464 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7x5w\" (UniqueName: \"kubernetes.io/projected/a10e4998-0b1d-4041-a022-9e36300936fa-kube-api-access-n7x5w\") pod \"volume-data-source-validator-58c9f5f46d-j8lxw\" (UID: \"a10e4998-0b1d-4041-a022-9e36300936fa\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j8lxw" Nov 04 05:26:59.273285 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.273242 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqqt4\" (UniqueName: \"kubernetes.io/projected/5704adc4-5d8d-439e-ae45-2ab73b06bf14-kube-api-access-vqqt4\") pod \"ingress-canary-767sh\" (UID: \"5704adc4-5d8d-439e-ae45-2ab73b06bf14\") " pod="openshift-ingress-canary/ingress-canary-767sh" Nov 04 05:26:59.318153 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.318128 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:26:59.341035 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.341010 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j8lxw" Nov 04 05:26:59.348615 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.348561 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xk9lw" Nov 04 05:26:59.377381 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.377353 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-767sh" Nov 04 05:26:59.488455 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.488431 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-pw5zp"] Nov 04 05:26:59.491945 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.491922 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-5zj4b"] Nov 04 05:26:59.495481 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:59.495458 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod523aff5c_ac2b_43be_8e13_792f44b2c14e.slice/crio-a70bed57c99af68bc7cf15bff43587864b448f233d985b7abf7a09f659f4162e WatchSource:0}: Error finding container a70bed57c99af68bc7cf15bff43587864b448f233d985b7abf7a09f659f4162e: Status 404 returned error can't find the container with id a70bed57c99af68bc7cf15bff43587864b448f233d985b7abf7a09f659f4162e Nov 04 05:26:59.512861 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.512807 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-h5ccq"] Nov 04 05:26:59.513281 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.513255 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:26:59.513390 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.513371 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:26:59.513580 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.513564 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Nov 04 05:26:59.514803 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.514782 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert podName:a35474ea-79b9-4d84-81b1-39db689685d2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:00.513603928 +0000 UTC m=+33.186742013 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pgdqn" (UID: "a35474ea-79b9-4d84-81b1-39db689685d2") : secret "networking-console-plugin-cert" not found Nov 04 05:26:59.514911 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.514888 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:26:59.514974 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.514919 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6469786477-m4mgb: secret "image-registry-tls" not found Nov 04 05:26:59.515029 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.514968 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls podName:00a06b1e-791a-4652-823c-b73672b7e4b2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:00.514952306 +0000 UTC m=+33.188090403 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls") pod "image-registry-6469786477-m4mgb" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2") : secret "image-registry-tls" not found Nov 04 05:26:59.519292 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:59.518331 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode10001da_2e9e_4b80_b6e0_0f1fca3df854.slice/crio-bf1d33ae64077954869a6356a46df86444a4af90371a39cab907c1f96f86f53e WatchSource:0}: Error finding container bf1d33ae64077954869a6356a46df86444a4af90371a39cab907c1f96f86f53e: Status 404 returned error can't find the container with id bf1d33ae64077954869a6356a46df86444a4af90371a39cab907c1f96f86f53e Nov 04 05:26:59.572459 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.572430 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl"] Nov 04 05:26:59.584202 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.584027 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9"] Nov 04 05:26:59.614182 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.613948 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:26:59.614182 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.614006 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:26:59.614182 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.614042 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:26:59.614182 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.614080 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:26:59.614182 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.614097 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:26:59.614182 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.614164 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Nov 04 05:26:59.614434 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.614193 2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Nov 04 05:26:59.614434 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.614211 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls podName:d998c080-e5b9-4de6-af9c-68a8c531257c nodeName:}" failed. No retries permitted until 2026-11-04 05:27:00.61419765 +0000 UTC m=+33.287335757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6nlwl" (UID: "d998c080-e5b9-4de6-af9c-68a8c531257c") : secret "samples-operator-tls" not found Nov 04 05:26:59.614434 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.614233 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:00.614218112 +0000 UTC m=+33.287356212 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : secret "router-metrics-certs-default" not found Nov 04 05:26:59.614434 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.614265 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:00.614258506 +0000 UTC m=+33.287396654 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : configmap references non-existent config key: service-ca.crt Nov 04 05:26:59.614434 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.614276 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Nov 04 05:26:59.614434 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.614298 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls podName:191dfb71-401c-494a-b7bb-cbc6c3273430 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:00.614291726 +0000 UTC m=+33.287429808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-hcsmg" (UID: "191dfb71-401c-494a-b7bb-cbc6c3273430") : secret "cluster-monitoring-operator-tls" not found Nov 04 05:26:59.614434 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.614307 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:26:59.614434 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.614315 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8f7d8c7c6-q48ph: secret "image-registry-tls" not found Nov 04 05:26:59.614434 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.614341 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls podName:aa931f6e-a107-4ca3-bb54-e89a91d0f67a nodeName:}" failed. No retries permitted until 2026-11-04 05:27:00.614335429 +0000 UTC m=+33.287473548 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls") pod "image-registry-8f7d8c7c6-q48ph" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a") : secret "image-registry-tls" not found Nov 04 05:26:59.640016 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.639944 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-qnfh8"] Nov 04 05:26:59.643244 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:59.643220 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58bf7d20_81e0_4dbc_85f2_32ded8f14e5f.slice/crio-186c505a55d86fd094ca455e6d35af705ded27fd77d9cdabb1a3a4f9c7167e3d WatchSource:0}: Error finding container 186c505a55d86fd094ca455e6d35af705ded27fd77d9cdabb1a3a4f9c7167e3d: Status 404 returned error can't find the container with id 186c505a55d86fd094ca455e6d35af705ded27fd77d9cdabb1a3a4f9c7167e3d Nov 04 05:26:59.705428 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.705398 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-b24ft"] Nov 04 05:26:59.708049 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:59.708025 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90fb287e_4af9_4e0b_919d_00713b37605c.slice/crio-981a3cfbef72505c1396f49c8f9064d2a5691eee3611d15ffbc3367c87ed44e7 WatchSource:0}: Error finding container 981a3cfbef72505c1396f49c8f9064d2a5691eee3611d15ffbc3367c87ed44e7: Status 404 returned error can't find the container with id 981a3cfbef72505c1396f49c8f9064d2a5691eee3611d15ffbc3367c87ed44e7 Nov 04 05:26:59.714856 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.714836 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:26:59.714986 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.714969 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:26:59.715032 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:26:59.715023 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls podName:886edd3a-6e51-4ccf-92a8-9d8a3d447449 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:00.715008395 +0000 UTC m=+33.388146491 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls") pod "dns-default-brcbz" (UID: "886edd3a-6e51-4ccf-92a8-9d8a3d447449") : secret "dns-default-metrics-tls" not found Nov 04 05:26:59.730617 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.730598 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j8lxw"] Nov 04 05:26:59.733458 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:59.733438 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda10e4998_0b1d_4041_a022_9e36300936fa.slice/crio-11c2917eaf5a3eab504ea4d368cf93f8f34e45e865290a9be71acdda3333af46 WatchSource:0}: Error finding container 11c2917eaf5a3eab504ea4d368cf93f8f34e45e865290a9be71acdda3333af46: Status 404 returned error can't find the container with id 11c2917eaf5a3eab504ea4d368cf93f8f34e45e865290a9be71acdda3333af46 Nov 04 05:26:59.750262 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.750241 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-xk9lw"] Nov 04 05:26:59.753619 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:59.753600 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25ddbfa_3e3e_49fd_870f_963d087f22a5.slice/crio-3c81de14011f735b2a9889357de534ae6f3e9e5456a2fbf9a044be15cf4936c6 WatchSource:0}: Error finding container 3c81de14011f735b2a9889357de534ae6f3e9e5456a2fbf9a044be15cf4936c6: Status 404 returned error can't find the container with id 3c81de14011f735b2a9889357de534ae6f3e9e5456a2fbf9a044be15cf4936c6 Nov 04 05:26:59.781557 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.781539 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-767sh"] Nov 04 05:26:59.784267 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:26:59.784240 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5704adc4_5d8d_439e_ae45_2ab73b06bf14.slice/crio-76d053db0d1efc284602d29e6f33d8750bac470e4072bc672484db13f154f21c WatchSource:0}: Error finding container 76d053db0d1efc284602d29e6f33d8750bac470e4072bc672484db13f154f21c: Status 404 returned error can't find the container with id 76d053db0d1efc284602d29e6f33d8750bac470e4072bc672484db13f154f21c Nov 04 05:26:59.882139 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.882081 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:26:59.882276 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.882260 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:26:59.930993 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.930971 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 05:26:59.931499 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.931025 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cbfrj\"" Nov 04 05:26:59.931499 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.931070 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 05:26:59.931499 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:26:59.931130 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-2psbf\"" Nov 04 05:27:00.070919 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.070893 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-767sh" event={"ID":"5704adc4-5d8d-439e-ae45-2ab73b06bf14","Type":"ContainerStarted","Data":"76d053db0d1efc284602d29e6f33d8750bac470e4072bc672484db13f154f21c"} Nov 04 05:27:00.071987 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.071955 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xk9lw" event={"ID":"b25ddbfa-3e3e-49fd-870f-963d087f22a5","Type":"ContainerStarted","Data":"3c81de14011f735b2a9889357de534ae6f3e9e5456a2fbf9a044be15cf4936c6"} Nov 04 05:27:00.073113 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.073085 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j8lxw" event={"ID":"a10e4998-0b1d-4041-a022-9e36300936fa","Type":"ContainerStarted","Data":"11c2917eaf5a3eab504ea4d368cf93f8f34e45e865290a9be71acdda3333af46"} Nov 04 05:27:00.074220 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.074198 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-b24ft" event={"ID":"90fb287e-4af9-4e0b-919d-00713b37605c","Type":"ContainerStarted","Data":"981a3cfbef72505c1396f49c8f9064d2a5691eee3611d15ffbc3367c87ed44e7"} Nov 04 05:27:00.075262 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.075244 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" event={"ID":"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f","Type":"ContainerStarted","Data":"186c505a55d86fd094ca455e6d35af705ded27fd77d9cdabb1a3a4f9c7167e3d"} Nov 04 05:27:00.076289 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.076255 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" event={"ID":"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04","Type":"ContainerStarted","Data":"b45574024cbf01b45820b72330280a0aa370363457b75cfe34e03fd6361d4ee0"} Nov 04 05:27:00.077253 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.077230 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" event={"ID":"d26b56ed-c6e3-4321-8044-d29e5fe081c8","Type":"ContainerStarted","Data":"48e160cc3b3d558da0739bcd3b1d4cbbaaec8d58904d767631cb7d25bf6d3f7d"} Nov 04 05:27:00.078394 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.078373 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-5zj4b" event={"ID":"523aff5c-ac2b-43be-8e13-792f44b2c14e","Type":"ContainerStarted","Data":"14d8e9595c0cb4796dc59930747e8a8d03dc99b219b9a6a79d71fcc37c80f6fa"} Nov 04 05:27:00.078514 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.078409 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-5zj4b" event={"ID":"523aff5c-ac2b-43be-8e13-792f44b2c14e","Type":"ContainerStarted","Data":"a70bed57c99af68bc7cf15bff43587864b448f233d985b7abf7a09f659f4162e"} Nov 04 05:27:00.079628 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.079610 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-pw5zp" event={"ID":"2e2ba652-6141-4984-8384-51f9720a02ef","Type":"ContainerStarted","Data":"563fdeec9a9cc0b7ea21ce9b5c0d9de9ae0b0d8b6c6c14e7c6a987c7757a3803"} Nov 04 05:27:00.079728 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.079645 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-pw5zp" event={"ID":"2e2ba652-6141-4984-8384-51f9720a02ef","Type":"ContainerStarted","Data":"901917c944bd1d7388e73ab52329720f1ebfb593f14f2276d89bff626841fcb2"} Nov 04 05:27:00.080880 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.080859 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-h5ccq" event={"ID":"e10001da-2e9e-4b80-b6e0-0f1fca3df854","Type":"ContainerStarted","Data":"0fe11ad4b9c3f41f23dc2d1b204a3b4703dec5a001ccdc74263defcb603332ae"} Nov 04 05:27:00.080968 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.080893 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-h5ccq" event={"ID":"e10001da-2e9e-4b80-b6e0-0f1fca3df854","Type":"ContainerStarted","Data":"bf1d33ae64077954869a6356a46df86444a4af90371a39cab907c1f96f86f53e"} Nov 04 05:27:00.521180 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.521154 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:00.521337 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.521225 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:27:00.521337 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.521302 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:27:00.521337 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.521320 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6469786477-m4mgb: secret "image-registry-tls" not found Nov 04 05:27:00.521447 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.521368 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls podName:00a06b1e-791a-4652-823c-b73672b7e4b2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:02.521352652 +0000 UTC m=+35.194490752 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls") pod "image-registry-6469786477-m4mgb" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2") : secret "image-registry-tls" not found Nov 04 05:27:00.521447 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.521373 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Nov 04 05:27:00.521447 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.521420 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert podName:a35474ea-79b9-4d84-81b1-39db689685d2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:02.521406527 +0000 UTC m=+35.194544635 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pgdqn" (UID: "a35474ea-79b9-4d84-81b1-39db689685d2") : secret "networking-console-plugin-cert" not found Nov 04 05:27:00.622032 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.621963 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:00.622032 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.621992 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:00.622188 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.622079 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:27:00.622188 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.622092 2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Nov 04 05:27:00.622188 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.622098 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8f7d8c7c6-q48ph: secret "image-registry-tls" not found Nov 04 05:27:00.622188 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.622110 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:00.622188 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.622141 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:02.622125904 +0000 UTC m=+35.295264003 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : secret "router-metrics-certs-default" not found Nov 04 05:27:00.622188 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.622169 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls podName:aa931f6e-a107-4ca3-bb54-e89a91d0f67a nodeName:}" failed. No retries permitted until 2026-11-04 05:27:02.62215962 +0000 UTC m=+35.295297719 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls") pod "image-registry-8f7d8c7c6-q48ph" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a") : secret "image-registry-tls" not found Nov 04 05:27:00.622188 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.622188 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:02.622177248 +0000 UTC m=+35.295315350 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : configmap references non-existent config key: service-ca.crt Nov 04 05:27:00.622545 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.622234 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:00.622545 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.622279 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:27:00.622545 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.622367 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Nov 04 05:27:00.622545 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.622402 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls podName:d998c080-e5b9-4de6-af9c-68a8c531257c nodeName:}" failed. No retries permitted until 2026-11-04 05:27:02.622393427 +0000 UTC m=+35.295531523 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6nlwl" (UID: "d998c080-e5b9-4de6-af9c-68a8c531257c") : secret "samples-operator-tls" not found Nov 04 05:27:00.622545 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.622367 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Nov 04 05:27:00.622545 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.622426 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls podName:191dfb71-401c-494a-b7bb-cbc6c3273430 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:02.622421612 +0000 UTC m=+35.295559707 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-hcsmg" (UID: "191dfb71-401c-494a-b7bb-cbc6c3273430") : secret "cluster-monitoring-operator-tls" not found Nov 04 05:27:00.722980 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.722822 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:00.722980 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.722878 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-h5ccq" podStartSLOduration=439.722865146 podStartE2EDuration="7m19.722865146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:19:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:00.722193021 +0000 UTC m=+33.395331127" watchObservedRunningTime="2026-11-04 05:27:00.722865146 +0000 UTC m=+33.396003252" Nov 04 05:27:00.722980 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.722974 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:00.723240 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.723023 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls podName:886edd3a-6e51-4ccf-92a8-9d8a3d447449 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:02.723011339 +0000 UTC m=+35.396149422 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls") pod "dns-default-brcbz" (UID: "886edd3a-6e51-4ccf-92a8-9d8a3d447449") : secret "dns-default-metrics-tls" not found Nov 04 05:27:00.823472 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.823447 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:27:00.823472 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.823477 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:27:00.823941 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.823626 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Nov 04 05:27:00.823941 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:00.823781 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs podName:4a509700-2c8d-4113-9121-3ec287223f70 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:32.823767739 +0000 UTC m=+65.496905826 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs") pod "network-metrics-daemon-55qcs" (UID: "4a509700-2c8d-4113-9121-3ec287223f70") : secret "metrics-daemon-secret" not found Nov 04 05:27:00.826108 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.826083 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:27:00.830820 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.830768 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-5zj4b" podStartSLOduration=439.830751982 podStartE2EDuration="7m19.830751982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:19:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:00.828837665 +0000 UTC m=+33.501975772" watchObservedRunningTime="2026-11-04 05:27:00.830751982 +0000 UTC m=+33.503890089" Nov 04 05:27:00.837365 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.837190 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:27:00.925237 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.925201 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbx8t\" (UniqueName: \"kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t\") pod \"network-check-target-27hf6\" (UID: \"4a5449c3-4f21-44d2-9d10-9a0956aefdcf\") " pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:27:00.949411 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:00.949382 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbx8t\" (UniqueName: \"kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t\") pod \"network-check-target-27hf6\" (UID: \"4a5449c3-4f21-44d2-9d10-9a0956aefdcf\") " pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:27:01.079676 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:01.079630 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:27:01.235453 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:01.235385 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-pw5zp" podStartSLOduration=440.235365125 podStartE2EDuration="7m20.235365125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:19:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:00.937889496 +0000 UTC m=+33.611027629" watchObservedRunningTime="2026-11-04 05:27:01.235365125 +0000 UTC m=+33.908503231" Nov 04 05:27:01.236310 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:01.236285 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nk2gk"] Nov 04 05:27:01.476482 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:01.476453 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-27hf6"] Nov 04 05:27:01.479332 ip-10-0-133-143 kubenswrapper[2573]: W1104 05:27:01.479274 2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a5449c3_4f21_44d2_9d10_9a0956aefdcf.slice/crio-1291829ddcc5c473402ff9425d57f0601a0cb87b7d8769c787d41a3acdd96d64 WatchSource:0}: Error finding container 1291829ddcc5c473402ff9425d57f0601a0cb87b7d8769c787d41a3acdd96d64: Status 404 returned error can't find the container with id 1291829ddcc5c473402ff9425d57f0601a0cb87b7d8769c787d41a3acdd96d64 Nov 04 05:27:02.093305 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:02.093265 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nk2gk" event={"ID":"26ae8a36-06bf-414e-b2bd-325033800b72","Type":"ContainerStarted","Data":"3266975edfbfb9106dd44dd9ed6ba2ba5a08329f9279468c1ca476ec9f277bf3"} Nov 04 05:27:02.097014 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:02.096985 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-27hf6" event={"ID":"4a5449c3-4f21-44d2-9d10-9a0956aefdcf","Type":"ContainerStarted","Data":"1291829ddcc5c473402ff9425d57f0601a0cb87b7d8769c787d41a3acdd96d64"} Nov 04 05:27:02.542500 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:02.542460 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:02.542704 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:02.542577 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:27:02.542704 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.542655 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:27:02.542704 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.542693 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6469786477-m4mgb: secret "image-registry-tls" not found Nov 04 05:27:02.542894 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.542750 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Nov 04 05:27:02.542894 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.542761 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls podName:00a06b1e-791a-4652-823c-b73672b7e4b2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:06.542744721 +0000 UTC m=+39.215882818 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls") pod "image-registry-6469786477-m4mgb" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2") : secret "image-registry-tls" not found Nov 04 05:27:02.542894 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.542801 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert podName:a35474ea-79b9-4d84-81b1-39db689685d2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:06.542788608 +0000 UTC m=+39.215926692 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pgdqn" (UID: "a35474ea-79b9-4d84-81b1-39db689685d2") : secret "networking-console-plugin-cert" not found Nov 04 05:27:02.643550 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:02.643514 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:02.643715 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:02.643572 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:02.643715 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:02.643607 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:27:02.643715 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:02.643635 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:02.643715 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:02.643653 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:02.643929 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.643808 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:27:02.643929 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.643815 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Nov 04 05:27:02.643929 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.643825 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8f7d8c7c6-q48ph: secret "image-registry-tls" not found Nov 04 05:27:02.643929 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.643860 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:06.6438384 +0000 UTC m=+39.316976505 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : configmap references non-existent config key: service-ca.crt Nov 04 05:27:02.643929 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.643903 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls podName:aa931f6e-a107-4ca3-bb54-e89a91d0f67a nodeName:}" failed. No retries permitted until 2026-11-04 05:27:06.643888486 +0000 UTC m=+39.317026574 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls") pod "image-registry-8f7d8c7c6-q48ph" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a") : secret "image-registry-tls" not found Nov 04 05:27:02.643929 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.643920 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls podName:d998c080-e5b9-4de6-af9c-68a8c531257c nodeName:}" failed. No retries permitted until 2026-11-04 05:27:06.64391021 +0000 UTC m=+39.317048335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6nlwl" (UID: "d998c080-e5b9-4de6-af9c-68a8c531257c") : secret "samples-operator-tls" not found Nov 04 05:27:02.644130 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.643929 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Nov 04 05:27:02.644130 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.643947 2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Nov 04 05:27:02.644130 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.643977 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls podName:191dfb71-401c-494a-b7bb-cbc6c3273430 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:06.643962302 +0000 UTC m=+39.317100385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-hcsmg" (UID: "191dfb71-401c-494a-b7bb-cbc6c3273430") : secret "cluster-monitoring-operator-tls" not found Nov 04 05:27:02.644130 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.644004 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:06.643989048 +0000 UTC m=+39.317127131 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : secret "router-metrics-certs-default" not found Nov 04 05:27:02.745103 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:02.745062 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:02.745268 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.745218 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:02.745332 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:02.745310 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls podName:886edd3a-6e51-4ccf-92a8-9d8a3d447449 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:06.745289737 +0000 UTC m=+39.418427833 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls") pod "dns-default-brcbz" (UID: "886edd3a-6e51-4ccf-92a8-9d8a3d447449") : secret "dns-default-metrics-tls" not found Nov 04 05:27:03.103123 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:03.103082 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j8lxw" event={"ID":"a10e4998-0b1d-4041-a022-9e36300936fa","Type":"ContainerStarted","Data":"653bca3ebf7d50e05d379193035f6d6a9883499bc1427d4a3586017323a2bb53"} Nov 04 05:27:03.214424 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:03.214249 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j8lxw" podStartSLOduration=404.060952561 podStartE2EDuration="6m47.214228895s" totalImagesPullingTime="3.153276334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:03.21192262 +0000 UTC m=+35.885060728" watchObservedRunningTime="2026-11-04 05:27:03.214228895 +0000 UTC m=+35.887367003" Nov 04 05:27:06.120836 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.120758 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" event={"ID":"d26b56ed-c6e3-4321-8044-d29e5fe081c8","Type":"ContainerStarted","Data":"737ff35e608128b78aee1b9c3e4f24c4adc55e252a075ccaac9529b7b60add89"} Nov 04 05:27:06.123928 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.123901 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-b24ft" event={"ID":"90fb287e-4af9-4e0b-919d-00713b37605c","Type":"ContainerStarted","Data":"1550427bdc24b8db1c10f40d5128980f9983c7bddecbcbae23bb1cf517a7a449"} Nov 04 05:27:06.126265 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.126247 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/0.log" Nov 04 05:27:06.126373 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.126278 2573 generic.go:356] "Generic (PLEG): container finished" podID="58bf7d20-81e0-4dbc-85f2-32ded8f14e5f" containerID="ec109870d77f86f6c090afd62765a40c9292ce8c525c6849a52b547191e124fa" exitCode=255 Nov 04 05:27:06.126373 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.126305 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" event={"ID":"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f","Type":"ContainerDied","Data":"ec109870d77f86f6c090afd62765a40c9292ce8c525c6849a52b547191e124fa"} Nov 04 05:27:06.126559 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.126543 2573 scope.go:122] "RemoveContainer" containerID="ec109870d77f86f6c090afd62765a40c9292ce8c525c6849a52b547191e124fa" Nov 04 05:27:06.229570 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.229514 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" podStartSLOduration=400.871748216 podStartE2EDuration="6m47.229498139s" totalImagesPullingTime="6.357749923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:20:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:06.229129257 +0000 UTC m=+38.902267395" watchObservedRunningTime="2026-11-04 05:27:06.229498139 +0000 UTC m=+38.902636244" Nov 04 05:27:06.459029 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.458694 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-b24ft" podStartSLOduration=405.729708483 podStartE2EDuration="6m51.458660013s" totalImagesPullingTime="5.72895153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:06.342153368 +0000 UTC m=+39.015291478" watchObservedRunningTime="2026-11-04 05:27:06.458660013 +0000 UTC m=+39.131798119" Nov 04 05:27:06.586687 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.586239 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:06.586687 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.586344 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:27:06.586687 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.586473 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Nov 04 05:27:06.586687 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.586533 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert podName:a35474ea-79b9-4d84-81b1-39db689685d2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.586515812 +0000 UTC m=+47.259653900 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pgdqn" (UID: "a35474ea-79b9-4d84-81b1-39db689685d2") : secret "networking-console-plugin-cert" not found Nov 04 05:27:06.586687 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.586604 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:27:06.586687 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.586615 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6469786477-m4mgb: secret "image-registry-tls" not found Nov 04 05:27:06.586687 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.586646 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls podName:00a06b1e-791a-4652-823c-b73672b7e4b2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.586636105 +0000 UTC m=+47.259774204 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls") pod "image-registry-6469786477-m4mgb" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2") : secret "image-registry-tls" not found Nov 04 05:27:06.687746 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.687702 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:27:06.687905 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.687775 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:06.687905 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.687805 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:06.687905 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.687886 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:06.688048 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.687947 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:06.688112 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.688100 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Nov 04 05:27:06.688172 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.688162 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls podName:d998c080-e5b9-4de6-af9c-68a8c531257c nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.688144806 +0000 UTC m=+47.361282904 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6nlwl" (UID: "d998c080-e5b9-4de6-af9c-68a8c531257c") : secret "samples-operator-tls" not found Nov 04 05:27:06.688240 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.688227 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Nov 04 05:27:06.688285 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.688271 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls podName:191dfb71-401c-494a-b7bb-cbc6c3273430 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.688254018 +0000 UTC m=+47.361392107 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-hcsmg" (UID: "191dfb71-401c-494a-b7bb-cbc6c3273430") : secret "cluster-monitoring-operator-tls" not found Nov 04 05:27:06.688341 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.688331 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:27:06.688376 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.688344 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8f7d8c7c6-q48ph: secret "image-registry-tls" not found Nov 04 05:27:06.688376 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.688374 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls podName:aa931f6e-a107-4ca3-bb54-e89a91d0f67a nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.688364404 +0000 UTC m=+47.361502492 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls") pod "image-registry-8f7d8c7c6-q48ph" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a") : secret "image-registry-tls" not found Nov 04 05:27:06.688453 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.688422 2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Nov 04 05:27:06.688485 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.688452 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.688443265 +0000 UTC m=+47.361581363 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : secret "router-metrics-certs-default" not found Nov 04 05:27:06.688517 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.688512 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.68850205 +0000 UTC m=+47.361640138 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : configmap references non-existent config key: service-ca.crt Nov 04 05:27:06.788752 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.788713 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:06.789036 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.788929 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:06.789036 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:06.788997 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls podName:886edd3a-6e51-4ccf-92a8-9d8a3d447449 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.788978591 +0000 UTC m=+47.462116689 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls") pod "dns-default-brcbz" (UID: "886edd3a-6e51-4ccf-92a8-9d8a3d447449") : secret "dns-default-metrics-tls" not found Nov 04 05:27:06.875353 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.875319 2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xfgjz"] Nov 04 05:27:06.898691 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.898632 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:06.935919 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.935855 2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xfgjz"] Nov 04 05:27:06.947196 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.947124 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 05:27:06.947302 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.947199 2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 05:27:06.947406 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.947390 2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ptjqz\"" Nov 04 05:27:06.989950 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.989926 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4qpn\" (UniqueName: \"kubernetes.io/projected/328415b5-7eda-429c-a74a-af29ba790b75-kube-api-access-p4qpn\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:06.990110 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.990087 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/328415b5-7eda-429c-a74a-af29ba790b75-crio-socket\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:06.990190 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.990162 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:06.990250 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.990232 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/328415b5-7eda-429c-a74a-af29ba790b75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:06.990316 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:06.990281 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/328415b5-7eda-429c-a74a-af29ba790b75-data-volume\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:07.091002 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.090966 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/328415b5-7eda-429c-a74a-af29ba790b75-crio-socket\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:07.091160 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.091039 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:07.091160 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.091081 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/328415b5-7eda-429c-a74a-af29ba790b75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:07.091160 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.091100 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/328415b5-7eda-429c-a74a-af29ba790b75-crio-socket\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:07.091160 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.091109 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/328415b5-7eda-429c-a74a-af29ba790b75-data-volume\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:07.091383 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:07.091184 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Nov 04 05:27:07.091383 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.091211 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4qpn\" (UniqueName: \"kubernetes.io/projected/328415b5-7eda-429c-a74a-af29ba790b75-kube-api-access-p4qpn\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:07.091383 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:07.091219 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:07.091383 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:07.091247 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/328415b5-7eda-429c-a74a-af29ba790b75-kube-rbac-proxy-cm podName:328415b5-7eda-429c-a74a-af29ba790b75 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:07.591228034 +0000 UTC m=+40.264366136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/328415b5-7eda-429c-a74a-af29ba790b75-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xfgjz" (UID: "328415b5-7eda-429c-a74a-af29ba790b75") : configmap "kube-rbac-proxy" not found Nov 04 05:27:07.091383 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:07.091270 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls podName:328415b5-7eda-429c-a74a-af29ba790b75 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:07.591261175 +0000 UTC m=+40.264399264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xfgjz" (UID: "328415b5-7eda-429c-a74a-af29ba790b75") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:07.091595 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.091432 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/328415b5-7eda-429c-a74a-af29ba790b75-data-volume\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:07.134271 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.134251 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/1.log" Nov 04 05:27:07.134909 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.134891 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/0.log" Nov 04 05:27:07.134976 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.134932 2573 generic.go:356] "Generic (PLEG): container finished" podID="58bf7d20-81e0-4dbc-85f2-32ded8f14e5f" containerID="28879b3159ff2859835187b11135533fdeb2de72dd6adcc016e7382ffad5bf8b" exitCode=255 Nov 04 05:27:07.135051 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.135032 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" event={"ID":"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f","Type":"ContainerDied","Data":"28879b3159ff2859835187b11135533fdeb2de72dd6adcc016e7382ffad5bf8b"} Nov 04 05:27:07.135102 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.135081 2573 scope.go:122] "RemoveContainer" containerID="ec109870d77f86f6c090afd62765a40c9292ce8c525c6849a52b547191e124fa" Nov 04 05:27:07.135387 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.135375 2573 scope.go:122] "RemoveContainer" containerID="28879b3159ff2859835187b11135533fdeb2de72dd6adcc016e7382ffad5bf8b" Nov 04 05:27:07.135614 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:07.135579 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-qnfh8_openshift-console-operator(58bf7d20-81e0-4dbc-85f2-32ded8f14e5f)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" podUID="58bf7d20-81e0-4dbc-85f2-32ded8f14e5f" Nov 04 05:27:07.137207 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.137185 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" event={"ID":"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04","Type":"ContainerStarted","Data":"593a8e6fdd494057081982351a782a2582bf25b804a1f2170082eea0914cc7e7"} Nov 04 05:27:07.153012 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.152986 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4qpn\" (UniqueName: \"kubernetes.io/projected/328415b5-7eda-429c-a74a-af29ba790b75-kube-api-access-p4qpn\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:07.356219 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.354970 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" podStartSLOduration=410.617275897 podStartE2EDuration="6m57.354951603s" totalImagesPullingTime="6.737675706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:07.354278946 +0000 UTC m=+40.027417051" watchObservedRunningTime="2026-11-04 05:27:07.354951603 +0000 UTC m=+40.028089710" Nov 04 05:27:07.595992 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.595955 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:07.596171 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:07.596038 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/328415b5-7eda-429c-a74a-af29ba790b75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:07.596171 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:07.596145 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:07.596288 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:07.596184 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Nov 04 05:27:07.596288 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:07.596197 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls podName:328415b5-7eda-429c-a74a-af29ba790b75 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:08.596183939 +0000 UTC m=+41.269322051 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xfgjz" (UID: "328415b5-7eda-429c-a74a-af29ba790b75") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:07.596288 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:07.596233 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/328415b5-7eda-429c-a74a-af29ba790b75-kube-rbac-proxy-cm podName:328415b5-7eda-429c-a74a-af29ba790b75 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:08.596218115 +0000 UTC m=+41.269356204 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/328415b5-7eda-429c-a74a-af29ba790b75-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xfgjz" (UID: "328415b5-7eda-429c-a74a-af29ba790b75") : configmap "kube-rbac-proxy" not found Nov 04 05:27:08.142239 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:08.142159 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/1.log" Nov 04 05:27:08.142693 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:08.142540 2573 scope.go:122] "RemoveContainer" containerID="28879b3159ff2859835187b11135533fdeb2de72dd6adcc016e7382ffad5bf8b" Nov 04 05:27:08.142812 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:08.142786 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-qnfh8_openshift-console-operator(58bf7d20-81e0-4dbc-85f2-32ded8f14e5f)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" podUID="58bf7d20-81e0-4dbc-85f2-32ded8f14e5f" Nov 04 05:27:08.144120 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:08.144087 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-27hf6" event={"ID":"4a5449c3-4f21-44d2-9d10-9a0956aefdcf","Type":"ContainerStarted","Data":"7509a3aaed23e701ff175cef6e79830bcea3255f6a36efeedaac493d8fb367a3"} Nov 04 05:27:08.144250 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:08.144211 2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:27:08.145907 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:08.145878 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-767sh" event={"ID":"5704adc4-5d8d-439e-ae45-2ab73b06bf14","Type":"ContainerStarted","Data":"0a4086fc113dc77aa9d2d76549635fc4ba6bbf17083d5dcf8b2f48a0ae6ec65a"} Nov 04 05:27:08.147569 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:08.147546 2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xk9lw" event={"ID":"b25ddbfa-3e3e-49fd-870f-963d087f22a5","Type":"ContainerStarted","Data":"601e69ff20a76d5ca018ac237efd2fc6702620b7881a36c587f1d19672cec3f1"} Nov 04 05:27:08.358440 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:08.358380 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xk9lw" podStartSLOduration=400.62048131 podStartE2EDuration="6m48.358364429s" totalImagesPullingTime="7.737883119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:08.357325504 +0000 UTC m=+41.030463612" watchObservedRunningTime="2026-11-04 05:27:08.358364429 +0000 UTC m=+41.031502579" Nov 04 05:27:08.465895 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:08.465844 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-27hf6" podStartSLOduration=34.275108473 podStartE2EDuration="40.465829522s" totalImagesPullingTime="6.190721049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:08.464416443 +0000 UTC m=+41.137554560" watchObservedRunningTime="2026-11-04 05:27:08.465829522 +0000 UTC m=+41.138967628" Nov 04 05:27:08.587053 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:08.586997 2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-767sh" podStartSLOduration=2.892456387 podStartE2EDuration="10.586982062s" totalImagesPullingTime="7.694525675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:08.585882513 +0000 UTC m=+41.259020638" watchObservedRunningTime="2026-11-04 05:27:08.586982062 +0000 UTC m=+41.260120177" Nov 04 05:27:08.606121 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:08.605452 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:08.606121 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:08.605513 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/328415b5-7eda-429c-a74a-af29ba790b75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:08.606121 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:08.605650 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Nov 04 05:27:08.606121 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:08.605720 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/328415b5-7eda-429c-a74a-af29ba790b75-kube-rbac-proxy-cm podName:328415b5-7eda-429c-a74a-af29ba790b75 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:10.605707476 +0000 UTC m=+43.278845571 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/328415b5-7eda-429c-a74a-af29ba790b75-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xfgjz" (UID: "328415b5-7eda-429c-a74a-af29ba790b75") : configmap "kube-rbac-proxy" not found Nov 04 05:27:08.606121 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:08.606054 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:08.606121 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:08.606096 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls podName:328415b5-7eda-429c-a74a-af29ba790b75 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:10.606085136 +0000 UTC m=+43.279223232 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xfgjz" (UID: "328415b5-7eda-429c-a74a-af29ba790b75") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:09.258792 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:09.258755 2573 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:27:09.259226 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:09.258822 2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:27:09.259226 ip-10-0-133-143 kubenswrapper[2573]: I1104 05:27:09.259166 2573 scope.go:122] "RemoveContainer" containerID="28879b3159ff2859835187b11135533fdeb2de72dd6adcc016e7382ffad5bf8b" Nov 04 05:27:09.259369 ip-10-0-133-143 kubenswrapper[2573]: E1104 05:27:09.259349 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-qnfh8_openshift-console-operator(58bf7d20-81e0-4dbc-85f2-32ded8f14e5f)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" podUID="58bf7d20-81e0-4dbc-85f2-32ded8f14e5f" Nov 04 05:27:09.932289 ip-10-0-133-143 systemd[1]: Stopping Kubernetes Kubelet... Nov 04 05:27:09.947042 ip-10-0-133-143 systemd[1]: kubelet.service: Deactivated successfully. Nov 04 05:27:09.947209 ip-10-0-133-143 systemd[1]: Stopped Kubernetes Kubelet. Nov 04 05:27:09.947273 ip-10-0-133-143 systemd[1]: kubelet.service: Consumed 2.883s CPU time. Nov 04 05:27:09.959234 ip-10-0-133-143 systemd[1]: Starting Kubernetes Kubelet... Nov 04 05:27:10.027882 ip-10-0-133-143 kubenswrapper[4859]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:27:10.027882 ip-10-0-133-143 kubenswrapper[4859]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 05:27:10.027882 ip-10-0-133-143 kubenswrapper[4859]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:27:10.027882 ip-10-0-133-143 kubenswrapper[4859]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:27:10.030728 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030632 4859 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:10.030728 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030722 4859 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:10.030728 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030726 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:10.030728 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030729 4859 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:10.030728 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030734 4859 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030737 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030741 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030743 4859 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030746 4859 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030749 4859 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030752 4859 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030755 4859 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030757 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030760 4859 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030763 4859 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030766 4859 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030770 4859 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030774 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030778 4859 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030780 4859 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030784 4859 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030787 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030790 4859 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:10.030932 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030793 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030796 4859 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030799 4859 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030801 4859 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030804 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030807 4859 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030809 4859 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030812 4859 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030814 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030817 4859 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030820 4859 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030823 4859 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030826 4859 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030829 4859 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030832 4859 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030834 4859 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030837 4859 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030840 4859 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030842 4859 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030845 4859 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:10.031409 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030847 4859 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030850 4859 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030852 4859 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030855 4859 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030858 4859 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030860 4859 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030863 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030865 4859 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030868 4859 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030871 4859 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030874 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030878 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030880 4859 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030883 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030886 4859 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030891 4859 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030894 4859 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030897 4859 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030899 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030902 4859 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:10.031933 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030905 4859 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030907 4859 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030910 4859 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030913 4859 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030915 4859 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030918 4859 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030921 4859 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030923 4859 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030926 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030929 4859 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030931 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030934 4859 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030937 4859 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030940 4859 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030943 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030945 4859 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030948 4859 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030951 4859 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030953 4859 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:10.032473 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030956 4859 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030959 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030962 4859 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030964 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030967 4859 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030969 4859 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030972 4859 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030975 4859 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030980 4859 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030983 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030985 4859 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030988 4859 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030991 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030993 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030996 4859 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.030999 4859 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031003 4859 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031007 4859 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031010 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031014 4859 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:10.032971 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031017 4859 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031021 4859 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031024 4859 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031027 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031823 4859 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031831 4859 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031834 4859 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031837 4859 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031840 4859 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031843 4859 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031846 4859 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031849 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031852 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031855 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031858 4859 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031861 4859 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031864 4859 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031867 4859 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031870 4859 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031873 4859 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:10.033505 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031876 4859 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031879 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031882 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031884 4859 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031887 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031890 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031892 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031895 4859 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031897 4859 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031900 4859 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031903 4859 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031906 4859 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031908 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031911 4859 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031913 4859 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031916 4859 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031919 4859 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031921 4859 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031924 4859 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031926 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:10.034008 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031929 4859 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031932 4859 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031934 4859 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031937 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031941 4859 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031944 4859 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031947 4859 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031954 4859 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031957 4859 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031959 4859 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031962 4859 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031965 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031968 4859 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031971 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031973 4859 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031976 4859 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031979 4859 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031982 4859 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031985 4859 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031988 4859 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:10.034503 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031990 4859 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031993 4859 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031995 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.031998 4859 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032001 4859 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032003 4859 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032006 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032008 4859 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032011 4859 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032014 4859 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032016 4859 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032019 4859 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032021 4859 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032024 4859 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032026 4859 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032029 4859 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032031 4859 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032034 4859 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032037 4859 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:10.035055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032040 4859 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032046 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032049 4859 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032052 4859 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032054 4859 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032057 4859 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032060 4859 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032064 4859 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032068 4859 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032071 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032074 4859 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032077 4859 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032080 4859 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032082 4859 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032085 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032087 4859 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032090 4859 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032093 4859 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032095 4859 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032098 4859 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:10.035524 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032101 4859 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032104 4859 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032106 4859 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032109 4859 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032112 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032114 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032117 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032122 4859 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032127 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032130 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032133 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032656 4859 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032677 4859 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032681 4859 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032684 4859 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032687 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032691 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032694 4859 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032696 4859 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:10.036042 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032699 4859 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032702 4859 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032705 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032708 4859 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032710 4859 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032713 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032716 4859 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032718 4859 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032721 4859 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032726 4859 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032729 4859 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032731 4859 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032735 4859 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032738 4859 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032740 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032744 4859 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032749 4859 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032752 4859 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032755 4859 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:10.036569 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032759 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032762 4859 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032766 4859 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032769 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032771 4859 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032774 4859 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032777 4859 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032779 4859 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032783 4859 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032786 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032789 4859 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032791 4859 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032794 4859 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032797 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032799 4859 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032802 4859 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032804 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032807 4859 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032809 4859 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032812 4859 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:10.037070 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032815 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032818 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032822 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032825 4859 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032828 4859 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032831 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032833 4859 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032836 4859 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032839 4859 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032842 4859 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032844 4859 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032847 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032849 4859 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032852 4859 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032855 4859 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032858 4859 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032860 4859 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032863 4859 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032866 4859 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:10.037555 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032869 4859 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032871 4859 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032874 4859 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032877 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032879 4859 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032882 4859 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032885 4859 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032887 4859 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032890 4859 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032892 4859 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032895 4859 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032897 4859 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032900 4859 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032902 4859 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032905 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032909 4859 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032912 4859 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032915 4859 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032917 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:10.038069 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032920 4859 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032923 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032925 4859 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032928 4859 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032930 4859 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032933 4859 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032936 4859 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032938 4859 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032941 4859 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032943 4859 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032946 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032949 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032951 4859 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032954 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032956 4859 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032959 4859 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032962 4859 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032965 4859 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032968 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:10.038532 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032971 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.032974 4859 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033066 4859 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033074 4859 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033079 4859 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033084 4859 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033088 4859 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033091 4859 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033096 4859 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033105 4859 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033109 4859 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033112 4859 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033116 4859 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033119 4859 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033122 4859 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033126 4859 flags.go:64] FLAG: --cgroup-root="" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033129 4859 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033132 4859 flags.go:64] FLAG: --client-ca-file="" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033135 4859 flags.go:64] FLAG: --cloud-provider="external" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033138 4859 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033143 4859 flags.go:64] FLAG: --cluster-domain="" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033145 4859 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033149 4859 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033152 4859 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 05:27:10.039007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033155 4859 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033164 4859 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033167 4859 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033170 4859 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033173 4859 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033176 4859 flags.go:64] FLAG: --contention-profiling="false" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033180 4859 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033183 4859 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033186 4859 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033189 4859 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033194 4859 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033197 4859 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033200 4859 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033203 4859 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033206 4859 flags.go:64] FLAG: --enable-server="true" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033209 4859 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033213 4859 flags.go:64] FLAG: --event-burst="100" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033218 4859 flags.go:64] FLAG: --event-qps="50" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033221 4859 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033224 4859 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033227 4859 flags.go:64] FLAG: --eviction-hard="" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033231 4859 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033234 4859 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033237 4859 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033240 4859 flags.go:64] FLAG: --eviction-soft="" Nov 04 05:27:10.039615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033243 4859 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033246 4859 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033249 4859 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033252 4859 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033256 4859 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033258 4859 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033262 4859 flags.go:64] FLAG: --feature-gates="" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033265 4859 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033269 4859 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033272 4859 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033275 4859 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033278 4859 flags.go:64] FLAG: --healthz-port="10248" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033281 4859 flags.go:64] FLAG: --help="false" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033284 4859 flags.go:64] FLAG: --hostname-override="ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033288 4859 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033291 4859 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033294 4859 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033298 4859 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033302 4859 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033304 4859 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033307 4859 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033310 4859 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033315 4859 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 05:27:10.040270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033318 4859 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033323 4859 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033326 4859 flags.go:64] FLAG: --kube-reserved="" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033329 4859 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033332 4859 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033335 4859 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033338 4859 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033341 4859 flags.go:64] FLAG: --lock-file="" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033344 4859 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033347 4859 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033350 4859 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033355 4859 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033358 4859 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033361 4859 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033364 4859 flags.go:64] FLAG: --logging-format="text" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033367 4859 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033371 4859 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033374 4859 flags.go:64] FLAG: --manifest-url="" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033377 4859 flags.go:64] FLAG: --manifest-url-header="" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033381 4859 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033385 4859 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033389 4859 flags.go:64] FLAG: --max-pods="110" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033392 4859 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033395 4859 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033398 4859 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 05:27:10.040867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033401 4859 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033404 4859 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033407 4859 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033410 4859 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033418 4859 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033421 4859 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033425 4859 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033428 4859 flags.go:64] FLAG: --pod-cidr="" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033433 4859 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033436 4859 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033439 4859 flags.go:64] FLAG: --pods-per-core="0" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033442 4859 flags.go:64] FLAG: --port="10250" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033445 4859 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033448 4859 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-056c087ce19aea98d" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033451 4859 flags.go:64] FLAG: --qos-reserved="" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033454 4859 flags.go:64] FLAG: --read-only-port="10255" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033457 4859 flags.go:64] FLAG: --register-node="true" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033461 4859 flags.go:64] FLAG: --register-with-taints="" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033465 4859 flags.go:64] FLAG: --registry-burst="10" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033468 4859 flags.go:64] FLAG: --registry-qps="5" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033471 4859 flags.go:64] FLAG: --reserved-cpus="" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033474 4859 flags.go:64] FLAG: --reserved-memory="" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033477 4859 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033481 4859 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033484 4859 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033487 4859 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 05:27:10.041559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033490 4859 flags.go:64] FLAG: --runonce="false" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033493 4859 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033496 4859 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033499 4859 flags.go:64] FLAG: --seccomp-default="false" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033502 4859 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033505 4859 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033509 4859 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033512 4859 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033515 4859 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033518 4859 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033521 4859 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033525 4859 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033529 4859 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033532 4859 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033536 4859 flags.go:64] FLAG: --system-cgroups="" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033539 4859 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033545 4859 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033548 4859 flags.go:64] FLAG: --tls-cert-file="" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033551 4859 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033555 4859 flags.go:64] FLAG: --tls-min-version="" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033558 4859 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033561 4859 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033564 4859 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033567 4859 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033570 4859 flags.go:64] FLAG: --v="2" Nov 04 05:27:10.042253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033574 4859 flags.go:64] FLAG: --version="false" Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033578 4859 flags.go:64] FLAG: --vmodule="" Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033583 4859 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.033586 4859 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033756 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033761 4859 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033765 4859 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033768 4859 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033771 4859 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033774 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033777 4859 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033779 4859 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033782 4859 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033785 4859 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033788 4859 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033790 4859 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033793 4859 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033796 4859 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033799 4859 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033803 4859 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033806 4859 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:10.043158 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033810 4859 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033813 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033815 4859 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033818 4859 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033821 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033824 4859 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033826 4859 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033830 4859 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033834 4859 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033837 4859 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033839 4859 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033842 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033845 4859 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033847 4859 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033850 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033853 4859 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033856 4859 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033859 4859 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033862 4859 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:10.044011 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033865 4859 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033867 4859 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033870 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033872 4859 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033875 4859 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033878 4859 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033881 4859 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033884 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033887 4859 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033889 4859 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033892 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033896 4859 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033899 4859 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033903 4859 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033906 4859 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033908 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033911 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033913 4859 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033916 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033919 4859 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:10.044626 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033921 4859 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033924 4859 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033926 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033929 4859 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033932 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033935 4859 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033937 4859 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033940 4859 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033943 4859 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033945 4859 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033950 4859 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033954 4859 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033957 4859 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033960 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033962 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033965 4859 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033968 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033971 4859 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033973 4859 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:10.045273 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033976 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033979 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033982 4859 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033984 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033989 4859 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033991 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033995 4859 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.033998 4859 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034000 4859 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034003 4859 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034006 4859 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034009 4859 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034011 4859 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034014 4859 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034017 4859 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034020 4859 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034023 4859 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034026 4859 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034028 4859 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:10.045891 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034031 4859 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:10.046365 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034034 4859 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:10.046365 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034036 4859 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:10.046365 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034039 4859 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:10.046365 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034041 4859 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:10.046365 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034044 4859 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:10.046365 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034047 4859 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:10.046365 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034049 4859 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:10.046365 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034052 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:10.046365 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034054 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:10.046365 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034057 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:10.046365 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034060 4859 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:10.046365 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.034086 4859 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:27:10.046365 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034127 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:10.046365 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034132 4859 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:10.046365 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034135 4859 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:10.046365 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034138 4859 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034142 4859 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034146 4859 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034148 4859 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034153 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034155 4859 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034158 4859 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034161 4859 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034164 4859 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034166 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034169 4859 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034172 4859 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034175 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034177 4859 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034180 4859 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034183 4859 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034185 4859 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034188 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034190 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034193 4859 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:10.047063 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034196 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034198 4859 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034201 4859 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034203 4859 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034206 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034209 4859 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034211 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034214 4859 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034217 4859 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034220 4859 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034222 4859 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034225 4859 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034229 4859 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034235 4859 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034238 4859 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034242 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034246 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034249 4859 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034252 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:10.047888 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034255 4859 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034257 4859 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034260 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034263 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034266 4859 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034268 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034271 4859 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034274 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034276 4859 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034279 4859 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034282 4859 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034284 4859 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034287 4859 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034289 4859 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034292 4859 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034295 4859 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034297 4859 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034300 4859 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034302 4859 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:10.048520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034306 4859 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034309 4859 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034311 4859 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034314 4859 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034316 4859 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034319 4859 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034322 4859 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034326 4859 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034329 4859 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034331 4859 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034336 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034338 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034341 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034344 4859 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034347 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034349 4859 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034352 4859 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034355 4859 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034357 4859 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034360 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034363 4859 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:10.049135 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034365 4859 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034368 4859 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034371 4859 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034373 4859 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034376 4859 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034379 4859 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034381 4859 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034384 4859 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034387 4859 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034389 4859 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034392 4859 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034394 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034397 4859 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034400 4859 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034402 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034405 4859 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034408 4859 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034412 4859 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034417 4859 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034420 4859 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:10.049650 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034423 4859 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034428 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034431 4859 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.034434 4859 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.038167 4859 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.038183 4859 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.038200 4859 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.038205 4859 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038261 4859 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038266 4859 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038269 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038272 4859 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038275 4859 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038278 4859 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038281 4859 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038284 4859 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038287 4859 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038289 4859 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038292 4859 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:10.050161 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038294 4859 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038297 4859 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038300 4859 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038303 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038307 4859 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038311 4859 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038314 4859 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038317 4859 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038320 4859 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038323 4859 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038325 4859 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038328 4859 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038331 4859 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038333 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038336 4859 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038339 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038342 4859 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038344 4859 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038353 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:10.050614 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038356 4859 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038358 4859 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038360 4859 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038363 4859 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038366 4859 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038368 4859 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038371 4859 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038373 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038376 4859 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038379 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038381 4859 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038383 4859 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038386 4859 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038388 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038391 4859 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038393 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038396 4859 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038398 4859 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038401 4859 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038403 4859 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:10.051085 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038406 4859 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038409 4859 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038412 4859 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038416 4859 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038420 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038424 4859 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038427 4859 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038430 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038433 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038435 4859 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038438 4859 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038441 4859 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038449 4859 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038451 4859 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038454 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038457 4859 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038459 4859 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038462 4859 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038464 4859 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:10.051658 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038467 4859 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038470 4859 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038472 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038475 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038477 4859 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038480 4859 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038482 4859 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038485 4859 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038487 4859 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038490 4859 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038493 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038495 4859 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038498 4859 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038500 4859 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038503 4859 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038506 4859 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038508 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038512 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038514 4859 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038517 4859 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:10.052331 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038520 4859 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:10.052981 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038522 4859 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:10.052981 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038525 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:10.052981 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038527 4859 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:10.052981 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038530 4859 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:10.052981 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038533 4859 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:10.052981 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038541 4859 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:10.052981 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038544 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:10.052981 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038546 4859 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:10.052981 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038549 4859 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:10.052981 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038552 4859 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:10.052981 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038554 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:10.052981 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038557 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:10.052981 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038559 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:10.052981 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038562 4859 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:10.052981 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038564 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:10.052981 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038566 4859 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:10.053506 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.038593 4859 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:27:10.053506 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038786 4859 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:10.053506 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038791 4859 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:10.053506 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038794 4859 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:10.053506 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038797 4859 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:10.053506 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038800 4859 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:10.053506 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038803 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:10.053506 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038806 4859 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:10.053506 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038809 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:10.053506 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038812 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:10.053506 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038815 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:10.053506 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038817 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:10.053506 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038820 4859 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:10.053506 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038822 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:10.053506 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038825 4859 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:10.053506 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038827 4859 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038831 4859 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038833 4859 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038836 4859 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038838 4859 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038841 4859 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038844 4859 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038852 4859 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038855 4859 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038857 4859 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038860 4859 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038862 4859 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038865 4859 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038867 4859 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038870 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038872 4859 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038875 4859 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038877 4859 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038880 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038883 4859 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:10.053914 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038885 4859 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038888 4859 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038890 4859 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038893 4859 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038896 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038898 4859 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038901 4859 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038903 4859 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038906 4859 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038908 4859 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038911 4859 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038913 4859 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038916 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038919 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038922 4859 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038924 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038927 4859 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038930 4859 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038934 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038937 4859 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:10.054376 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038946 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038949 4859 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038952 4859 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038955 4859 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038959 4859 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038962 4859 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038965 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038968 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038971 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038974 4859 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038977 4859 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038979 4859 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038982 4859 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038984 4859 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038987 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038989 4859 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038991 4859 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038994 4859 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038997 4859 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:10.055146 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.038999 4859 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039002 4859 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039004 4859 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039012 4859 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039016 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039019 4859 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039022 4859 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039024 4859 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039027 4859 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039030 4859 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039032 4859 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039035 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039037 4859 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039040 4859 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039048 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039051 4859 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039054 4859 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039056 4859 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039059 4859 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:10.055604 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039061 4859 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:10.056110 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039064 4859 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:10.056110 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039067 4859 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:10.056110 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039069 4859 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:10.056110 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039072 4859 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:10.056110 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039075 4859 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:10.056110 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039077 4859 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:10.056110 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039080 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:10.056110 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039082 4859 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:10.056110 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039085 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:10.056110 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039087 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:10.056110 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039090 4859 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:10.056110 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039092 4859 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:10.056110 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039095 4859 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:10.056110 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.039118 4859 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:27:10.056110 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039154 4859 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:10.056110 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039158 4859 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039161 4859 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039164 4859 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039167 4859 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039169 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039172 4859 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039174 4859 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039177 4859 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039180 4859 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039182 4859 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039185 4859 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039187 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039200 4859 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039203 4859 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039205 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039208 4859 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039211 4859 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039213 4859 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039216 4859 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:10.056502 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039218 4859 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039221 4859 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039223 4859 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039226 4859 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039229 4859 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039232 4859 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039234 4859 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039236 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039239 4859 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039242 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039244 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039247 4859 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039249 4859 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039252 4859 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039254 4859 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039257 4859 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039259 4859 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039262 4859 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039265 4859 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039267 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:10.057050 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039270 4859 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039272 4859 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039275 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039277 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039280 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039282 4859 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039290 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039293 4859 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039296 4859 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039298 4859 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039301 4859 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039303 4859 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039306 4859 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039309 4859 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039311 4859 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039314 4859 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039316 4859 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039319 4859 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039321 4859 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:10.057520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039324 4859 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039326 4859 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039329 4859 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039331 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039334 4859 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039336 4859 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039341 4859 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039344 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039346 4859 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039349 4859 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039353 4859 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039355 4859 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039358 4859 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039360 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039363 4859 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039365 4859 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039368 4859 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039370 4859 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039373 4859 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039376 4859 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:10.057991 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039384 4859 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039387 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039390 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039392 4859 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039395 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039397 4859 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039401 4859 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039405 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039408 4859 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039411 4859 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039414 4859 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039417 4859 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039420 4859 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039425 4859 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039429 4859 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039432 4859 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039435 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039438 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039443 4859 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:10.058535 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039445 4859 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:10.059025 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039448 4859 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:10.059025 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039450 4859 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:10.059025 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039453 4859 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:10.059025 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039455 4859 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:10.059025 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039460 4859 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:10.059025 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039462 4859 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:10.059025 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:10.039465 4859 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:10.059025 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.039520 4859 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 05:27:10.059025 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.040507 4859 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Nov 04 05:27:10.059025 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.040551 4859 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Nov 04 05:27:10.059025 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.040763 4859 server.go:1014] "Starting client certificate rotation" Nov 04 05:27:10.059025 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.040897 4859 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 05:27:10.059025 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.040946 4859 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 05:21:27 +0000 UTC" deadline="2028-07-14 18:44:59.795898854 +0000 UTC" Nov 04 05:27:10.059025 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.040974 4859 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14845h17m49.754926583s" Nov 04 05:27:10.059025 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.041259 4859 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 05:27:10.059025 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.042821 4859 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 05:27:10.059688 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.045927 4859 log.go:25] "Validated CRI v1 runtime API" Nov 04 05:27:10.059688 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.047893 4859 log.go:25] "Validated CRI v1 image API" Nov 04 05:27:10.059688 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.048606 4859 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 05:27:10.059688 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.050462 4859 fs.go:135] Filesystem UUIDs: map[70bf3a10-78bc-441e-a01c-7110aac0c462:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 7fe47483-ccd5-4130-8ca7-e45ccb05ef7c:/dev/nvme0n1p3] Nov 04 05:27:10.059946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.050477 4859 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0036c1fcf22adf4a202a1b3d09c1f9e85b4fa8d40e0ff508318537d5859f6499/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0036c1fcf22adf4a202a1b3d09c1f9e85b4fa8d40e0ff508318537d5859f6499/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/03b38d7f8d93e469a723b47425c44e0922c67780ce06bb599cf5ee5089fd17ca/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/03b38d7f8d93e469a723b47425c44e0922c67780ce06bb599cf5ee5089fd17ca/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/11c2917eaf5a3eab504ea4d368cf93f8f34e45e865290a9be71acdda3333af46/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/11c2917eaf5a3eab504ea4d368cf93f8f34e45e865290a9be71acdda3333af46/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1291829ddcc5c473402ff9425d57f0601a0cb87b7d8769c787d41a3acdd96d64/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1291829ddcc5c473402ff9425d57f0601a0cb87b7d8769c787d41a3acdd96d64/userdata/shm major:0 minor:290 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/186c505a55d86fd094ca455e6d35af705ded27fd77d9cdabb1a3a4f9c7167e3d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/186c505a55d86fd094ca455e6d35af705ded27fd77d9cdabb1a3a4f9c7167e3d/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3266975edfbfb9106dd44dd9ed6ba2ba5a08329f9279468c1ca476ec9f277bf3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3266975edfbfb9106dd44dd9ed6ba2ba5a08329f9279468c1ca476ec9f277bf3/userdata/shm major:0 minor:287 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3c81de14011f735b2a9889357de534ae6f3e9e5456a2fbf9a044be15cf4936c6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3c81de14011f735b2a9889357de534ae6f3e9e5456a2fbf9a044be15cf4936c6/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3f92655005734c04835a73a0a2c46997d2abe0b481a112c6395733a7731f21cd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3f92655005734c04835a73a0a2c46997d2abe0b481a112c6395733a7731f21cd/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/463c38893c854b23e9be888f730e7a64ecaba26981bc47571ba19fa0317bc9fa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/463c38893c854b23e9be888f730e7a64ecaba26981bc47571ba19fa0317bc9fa/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/48e160cc3b3d558da0739bcd3b1d4cbbaaec8d58904d767631cb7d25bf6d3f7d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/48e160cc3b3d558da0739bcd3b1d4cbbaaec8d58904d767631cb7d25bf6d3f7d/userdata/shm major:0 minor:227 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/76d053db0d1efc284602d29e6f33d8750bac470e4072bc672484db13f154f21c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/76d053db0d1efc284602d29e6f33d8750bac470e4072bc672484db13f154f21c/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/788d9f6ddbe47b4b03358ed63dc914669c3f40a5b8023bb26a91a0681452fddc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/788d9f6ddbe47b4b03358ed63dc914669c3f40a5b8023bb26a91a0681452fddc/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8e46f3b116be4a32a514c6bd4dfe03c8f7a60ce85d4a42a1947d7652e289f25c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8e46f3b116be4a32a514c6bd4dfe03c8f7a60ce85d4a42a1947d7652e289f25c/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/901917c944bd1d7388e73ab52329720f1ebfb593f14f2276d89bff626841fcb2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/901917c944bd1d7388e73ab52329720f1ebfb593f14f2276d89bff626841fcb2/userdata/shm major:0 minor:208 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/981a3cfbef72505c1396f49c8f9064d2a5691eee3611d15ffbc3367c87ed44e7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/981a3cfbef72505c1396f49c8f9064d2a5691eee3611d15ffbc3367c87ed44e7/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a70bed57c99af68bc7cf15bff43587864b448f233d985b7abf7a09f659f4162e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a70bed57c99af68bc7cf15bff43587864b448f233d985b7abf7a09f659f4162e/userdata/shm major:0 minor:216 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b45574024cbf01b45820b72330280a0aa370363457b75cfe34e03fd6361d4ee0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b45574024cbf01b45820b72330280a0aa370363457b75cfe34e03fd6361d4ee0/userdata/shm major:0 minor:229 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/beca5780d1576f0b8b961482c174af7488c9be39a7136b68b4fad07dfa5673db/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/beca5780d1576f0b8b961482c174af7488c9be39a7136b68b4fad07dfa5673db/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bf1d33ae64077954869a6356a46df86444a4af90371a39cab907c1f96f86f53e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bf1d33ae64077954869a6356a46df86444a4af90371a39cab907c1f96f86f53e/userdata/shm major:0 minor:218 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c65039c243a532bb0b419e22a11d20186c65571e11c590f0f61540e727812975/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c65039c243a532bb0b419e22a11d20186c65571e11c590f0f61540e727812975/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c8bdb05cd3620d33090b4d863e9485be49c1e5fb7fc1bf8122b59451e19b15e4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c8bdb05cd3620d33090b4d863e9485be49c1e5fb7fc1bf8122b59451e19b15e4/userdata/shm major:0 minor:69 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e44c5924347099be8c386a1c359936f6810d6c950da0b78944869ce8d5e90a1c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e44c5924347099be8c386a1c359936f6810d6c950da0b78944869ce8d5e90a1c/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00a06b1e-791a-4652-823c-b73672b7e4b2/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/00a06b1e-791a-4652-823c-b73672b7e4b2/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00a06b1e-791a-4652-823c-b73672b7e4b2/volumes/kubernetes.io~projected/kube-api-access-ws4hd:{mountpoint:/var/lib/kubelet/pods/00a06b1e-791a-4652-823c-b73672b7e4b2/volumes/kubernetes.io~projected/kube-api-access-ws4hd major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00a06b1e-791a-4652-823c-b73672b7e4b2/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/00a06b1e-791a-4652-823c-b73672b7e4b2/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00a06b1e-791a-4652-823c-b73672b7e4b2/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/00a06b1e-791a-4652-823c-b73672b7e4b2/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/191dfb71-401c-494a-b7bb-cbc6c3273430/volumes/kubernetes.io~projected/kube-api-access-gfptj:{mountpoint:/var/lib/kubelet/pods/191dfb71-401c-494a-b7bb-cbc6c3273430/volumes/kubernetes.io~projected/kube-api-access-gfptj major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a566754-7023-408d-bb6a-3eb984ffa7e3/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/1a566754-7023-408d-bb6a-3eb984ffa7e3/volumes/kubernetes.io~secret/agent-certs major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/26ae8a36-06bf-414e-b2bd-325033800b72/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/26ae8a36-06bf-414e-b2bd-325033800b72/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2e2ba652-6141-4984-8384-51f9720a02ef/volumes/kubernetes.io~projected/kube-api-access-m6gpf:{mountpoint:/var/lib/kubelet/pods/2e2ba652-6141-4984-8384-51f9720a02ef/volumes/kubernetes.io~projected/kube-api-access-m6gpf major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/328415b5-7eda-429c-a74a-af29ba790b75/volumes/kubernetes.io~projected/kube-api-access-p4qpn:{mountpoint:/var/lib/kubelet/pods/328415b5-7eda-429c-a74a-af29ba790b75/volumes/kubernetes.io~projected/kube-api-access-p4qpn major:0 minor:306 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/38b751da-304f-4000-bf56-85e1d9aa8152/volumes/kubernetes.io~projected/kube-api-access-cpww4:{mountpoint:/var/lib/kubelet/pods/38b751da-304f-4000-bf56-85e1d9aa8152/volumes/kubernetes.io~projected/kube-api-access-cpww4 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/428ba86e-0dd8-4a0a-b899-c7e9d568487c/volumes/kubernetes.io~projected/kube-api-access-9s7c8:{mountpoint:/var/lib/kubelet/pods/428ba86e-0dd8-4a0a-b899-c7e9d568487c/volumes/kubernetes.io~projected/kube-api-access-9s7c8 major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a509700-2c8d-4113-9121-3ec287223f70/volumes/kubernetes.io~projected/kube-api-access-484lx:{mountpoint:/var/lib/kubelet/pods/4a509700-2c8d-4113-9121-3ec287223f70/volumes/kubernetes.io~projected/kube-api-access-484lx major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a5449c3-4f21-44d2-9d10-9a0956aefdcf/volumes/kubernetes.io~projected/kube-api-access-fbx8t:{mountpoint:/var/lib/kubelet/pods/4a5449c3-4f21-44d2-9d10-9a0956aefdcf/volumes/kubernetes.io~projected/kube-api-access-fbx8t major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/523aff5c-ac2b-43be-8e13-792f44b2c14e/volumes/kubernetes.io~projected/kube-api-access-c2flm:{mountpoint:/var/lib/kubelet/pods/523aff5c-ac2b-43be-8e13-792f44b2c14e/volumes/kubernetes.io~projected/kube-api-access-c2flm major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5704adc4-5d8d-439e-ae45-2ab73b06bf14/volumes/kubernetes.io~projected/kube-api-access-vqqt4:{mountpoint:/var/lib/kubelet/pods/5704adc4-5d8d-439e-ae45-2ab73b06bf14/volumes/kubernetes.io~projected/kube-api-access-vqqt4 major:0 minor:237 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5704adc4-5d8d-439e-ae45-2ab73b06bf14/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/5704adc4-5d8d-439e-ae45-2ab73b06bf14/volumes/kubernetes.io~secret/cert major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/volumes/kubernetes.io~projected/kube-api-access-xjngp:{mountpoint:/var/lib/kubelet/pods/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/volumes/kubernetes.io~projected/kube-api-access-xjngp major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/volumes/kubernetes.io~secret/serving-cert major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5e7c38cd-3943-484d-b433-cbec7bce35c6/volumes/kubernetes.io~projected/kube-api-access-qghpk:{mountpoint:/var/lib/kubelet/pods/5e7c38cd-3943-484d-b433-cbec7bce35c6/volumes/kubernetes.io~projected/kube-api-access-qghpk major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6cf77b4d-32f5-42db-851f-3466ecf583c9/volumes/kubernetes.io~projected/kube-api-access-8d5fj:{mountpoint:/var/lib/kubelet/pods/6cf77b4d-32f5-42db-851f-3466ecf583c9/volumes/kubernetes.io~projected/kube-api-access-8d5fj major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/72dd8eb9-265d-4761-a507-d91e29606698/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/72dd8eb9-265d-4761-a507-d91e29606698/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/72dd8eb9-265d-4761-a507-d91e29606698/volumes/kubernetes.io~projected/kube-api-access-trzdq:{mountpoint:/var/lib/kubelet/pods/72dd8eb9-265d-4761-a507-d91e29606698/volumes/kubernetes.io~projected/kube-api-access-trzdq major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/72dd8eb9-265d-4761-a507-d91e29606698/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/72dd8eb9-265d-4761-a507-d91e29606698/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/77f9e498-8338-4f89-ae13-e72e0108a602/volumes/kubernetes.io~projected/kube-api-access-2qw4k:{mountpoint:/var/lib/kubelet/pods/77f9e498-8338-4f89-ae13-e72e0108a602/volumes/kubernetes.io~projected/kube-api-access-2qw4k major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/886edd3a-6e51-4ccf-92a8-9d8a3d447449/volumes/kubernetes.io~projected/kube-api-access-hbg2j:{mountpoint:/var/lib/kubelet/pods/886edd3a-6e51-4ccf-92a8-9d8a3d447449/volumes/kubernetes.io~projected/kube-api-access-hbg2j major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8f7d857f-84f9-4bde-9b5a-66164c90377e/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/8f7d857f-84f9-4bde-9b5a-66164c90377e/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8f7d857f-84f9-4bde-9b5a-66164c90377e/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/8f7d857f-84f9-4bde-9b5a-66164c90377e/volumes/kubernetes.io~empty-dir/tmp major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8f7d857f-84f9-4bde-9b5a-66164c90377e/volumes/kubernetes.io~projected/kube-api-access-hjmwm:{mountpoint:/var/lib/kubelet/pods/8f7d857f-84f9-4bde-9b5a-66164c90377e/volumes/kubernetes.io~projected/kube-api-access-hjmwm major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/90fb287e-4af9-4e0b-919d-00713b37605c/volumes/kubernetes.io~projected/kube-api-access-m7488:{mountpoint:/var/lib/kubelet/pods/90fb287e-4af9-4e0b-919d-00713b37605c/volumes/kubernetes.io~projected/kube-api-access-m7488 major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/90fb287e-4af9-4e0b-919d-00713b37605c/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/90fb287e-4af9-4e0b-919d-00713b37605c/volumes/kubernetes.io~secret/serving-cert major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a10e4998-0b1d-4041-a022-9e36300936fa/volumes/kubernetes.io~projected/kube-api-access-n7x5w:{mountpoint:/var/lib/kubelet/pods/a10e4998-0b1d-4041-a022-9e36300936fa/volumes/kubernetes.io~projected/kube-api-access-n7x5w major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aa931f6e-a107-4ca3-bb54-e89a91d0f67a/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/aa931f6e-a107-4ca3-bb54-e89a91d0f67a/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aa931f6e-a107-4ca3-bb54-e89a91d0f67a/volumes/kubernetes.io~projected/kube-api-access-8prxh:{mountpoint:/var/lib/kubelet/pods/aa931f6e-a107-4ca3-bb54-e89a91d0f67a/volumes/kubernetes.io~projected/kube-api-access-8prxh major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aa931f6e-a107-4ca3-bb54-e89a91d0f67a/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/aa931f6e-a107-4ca3-bb54-e89a91d0f67a/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aa931f6e-a107-4ca3-bb54-e89a91d0f67a/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/aa931f6e-a107-4ca3-bb54-e89a91d0f67a/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b25ddbfa-3e3e-49fd-870f-963d087f22a5/volumes/kubernetes.io~projected/kube-api-access-5rvs5:{mountpoint:/var/lib/kubelet/pods/b25ddbfa-3e3e-49fd-870f-963d087f22a5/volumes/kubernetes.io~projected/kube-api-access-5rvs5 major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04/volumes/kubernetes.io~projected/kube-api-access-dtv2t:{mountpoint:/var/lib/kubelet/pods/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04/volumes/kubernetes.io~projected/kube-api-access-dtv2t major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04/volumes/kubernetes.io~secret/serving-cert major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c87f7dad-1181-4db5-b846-05d8fb0a33d5/volumes/kubernetes.io~projected/kube-api-access-tz2hq:{mountpoint:/var/lib/kubelet/pods/c87f7dad-1181-4db5-b846-05d8fb0a33d5/volumes/kubernetes.io~projected/kube-api-access-tz2hq major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c87f7dad-1181-4db5-b846-05d8fb0a33d5/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/c87f7dad-1181-4db5-b846-05d8fb0a33d5/volumes/kubernetes.io~secret/default-certificate major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c87f7dad-1181-4db5-b846-05d8fb0a33d5/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/c87f7dad-1181-4db5-b846-05d8fb0a33d5/volumes/kubernetes.io~secret/stats-auth major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d26b56ed-c6e3-4321-8044-d29e5fe081c8/volumes/kubernetes.io~projected/kube-api-access-xwj9c:{mountpoint:/var/lib/kubelet/pods/d26b56ed-c6e3-4321-8044-d29e5fe081c8/volumes/kubernetes.io~projected/kube-api-access-xwj9c major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d26b56ed-c6e3-4321-8044-d29e5fe081c8/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/d26b56ed-c6e3-4321-8044-d29e5fe081c8/volumes/kubernetes.io~secret/serving-cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d998c080-e5b9-4de6-af9c-68a8c531257c/volumes/kubernetes.io~projected/kube-api-access-5p6g8:{mountpoint:/var/lib/kubelet/pods/d998c080-e5b9-4de6-af9c-68a8c531257c/volumes/kubernetes.io~projected/kube-api-access-5p6g8 major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e10001da-2e9e-4b80-b6e0-0f1fca3df854/volumes/kubernetes.io~projected/kube-api-access-2fbtn:{mountpoint:/var/lib/kubelet/pods/e10001da-2e9e-4b80-b6e0-0f1fca3df854/volumes/kubernetes.io~projected/kube-api-access-2fbtn major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb/volumes/kubernetes.io~projected/kube-api-access-x87gt:{mountpoint:/var/lib/kubelet/pods/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb/volumes/kubernetes.io~projected/kube-api-access-x87gt major:0 minor:57 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/0a1671ce668988b9decf28bbb9a19bdfd1b04d3b20a9bcc520375ea1efb7c8b7/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/36d1a9fd0088705ac6de63bc8c10a1eaf950ad96535041f6baf79cf8d5cf4b97/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/8beea29c800d8460dcb3ec20a7c3a75696004c62e0e8bdbb292237d84db66b54/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/e2703cf82232d644e3603492e5b25e1f9521c34bcb97f9c167a91a2882c770c2/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/405783477a99698232437137c898cd0a8a1be1f57c98f7755c04f9d85d65a6d4/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/96992e9f7b34736d659833cdd8973e0606fea95923abc6a930ca556ce679a48b/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-122:{mountpoint:/var/lib/containers/storage/overlay/951949050bcacfd45b0d470f5aefb763dc92f6277fa42ab115e00aa18989a791/merged major:0 minor:122 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/62d8579218e550a7f3998c3ea9f7d17059ba9d013c2141b1d066517ef41aa2ca/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/13431a5e4ca22cac7a26c953e07ff92b1fd102117c88182e2170dd52fb74b306/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/1bfbb3ab57bccf54dc73fc380528d7d8cc5ac5d693628b5b7617a019ac719307/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-135:{mountpoint:/var/lib/containers/storage/overlay/ae2e6ea080f859d631c644f7cbb226fcb90bde40cab19f53d7fd4df85f1ed803/merged major:0 minor:135 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/570c3414526b02a8cedf7a18e4e7f51abe55c14d58dc6f1c097ed45c67a15c4b/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/0a6ba63ea00a90ae89903eca0a80a5872a1a0c04f96fe9a85093de8c194a5d4b/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/d6fa4e4a384972e319f64153b5080acb7e16016cd254d96cfbcbd06ce6481cf3/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/4244386be4e9987ffa1d1b926795cdfe1c0103c14214d0644349c5102d393ca9/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-168:{mountpoint:/var/lib/containers/storage/overlay/235ae7f37808e0875e17babd4376dba0a3cdd8d3c8d63c9ddd96bdf7fe6ebace/merged major:0 minor:168 fsType:overlay blockSize:0} overlay_0-171:{mountpoint:/var/lib/containers/storage/overlay/efdfb0df37ab55269225567ffbac1e25750b09defd540c40797e80f081e12abe/merged major:0 minor:171 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/d3b45269be8f6c5e936d8c1f4393e35af215e100a23cd24dce31174eea04221e/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/86627651c481a9dc18021b34360fb0f0e83e4d743b2f0113fecaa7c352bf59cd/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/4a2ae3d11f747fd52af60484eddec1e298d2f9364bdd1c8e5438d2eabfd8b285/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/9a78cbb4a19292018d45d7bce074261da01fd0028ffe7dbc311e754455144292/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/045c09e8b3eb63134390d949ebe0c8876c834dd9f08dd978c8c62e15fde1797c/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/3ea367d0edc4903f6ccc19c0f87499229442035f43b3c9d3f31be8e423cf0be0/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/314ceb1e600b26c7f8daf011d18db83fa1838d5ac01e34a53dcf1955d4cb4f83/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/4a52198a1ea8bd3c8f7865df669c32873e4e0d927f3fae3af3c65fb8c17a807f/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/aff8e229e18eb50e4366390efd87a6e397a06119d67e9906d949f982403e9c5a/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/bcf8d12c8e10239a48382cea90f705558b4115dc135b4087ca8e6f99a5d7f009/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/22c0107687d46c2c3feedf01828a2fa9f31e8e0a0669fe0da6a17f7848ffb779/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/489593872f015fbb5917588f58df47226ebfb0989452da091d19527de7827a73/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/7dc1dc8824c2fb024bece9a032bf6804cb7e24c382a179d6de9b410ffdc187ec/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/bec1575c1e1785006b1dbb900be1ad45024769747066a0c674aac9335d3ae5d5/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-279:{mountpoint:/var/lib/containers/storage/overlay/729042f3a7a0afa0f54c1aca7a95fce747e00397c1cbddc3b3729c29c45e8a14/merged major:0 minor:279 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/0ddef5025d08c86f130b24f7a60ec918f586414808a2514346893b136937695a/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/97424b65ec4186c95155b8ac2f4f4fd34ea369d5810d715f512fc6d41baeb99a/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-292:{mountpoint:/var/lib/containers/storage/overlay/57df0b70a1a7c13f1b00f350516a0786a7f2bf3714aa81656a590c4a48a2d573/merged major:0 minor:292 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/220847b38d54dbc5150b4c1916122551e4ed7e67ca0a7ed90f2314bd2bd6ad5f/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/e7c5b4d4da2f1c0682865f82858c38f18d0cb5ce179a1a71e5eba407f1b4fb26/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/d967cae78bacb1234b53e62428f39a721057704552689f7ba5b9f2be3734f40e/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/a9b674ec192374028b5755ffff7be60fd73b3f27eaced9498c632f91472c38b9/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-315:{mountpoint:/var/lib/containers/storage/overlay/cb430ca0e6f4bf27e030a5a032b8f4425483a5c61ed69ab7a1dacd298ad59a5b/merged major:0 minor:315 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/2613b42e56e621c7f696104e9df0e02cfa138b4754dc2b966a188ff27b75e433/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-319:{mountpoint:/var/lib/containers/storage/overlay/295ba2cc12f810695d2b98e805aa98de33340cc1b682f46650200474b74d41c8/merged major:0 minor:319 fsType:overlay blockSize:0} overlay_0-324:{mountpoint:/var/lib/containers/storage/overlay/e961bb5b5d1fe779199f027d0c77424c4cc5129cb7a47192462d7e7f1ffd17c9/merged major:0 minor:324 fsType:overlay blockSize:0} overlay_0-335:{mountpoint:/var/lib/containers/storage/overlay/fbb7b2535035b4bcdd5e3ed6354ee508f2bf04433d3d12b9b17c3303d4104d2a/merged major:0 minor:335 fsType:overlay blockSize:0} overlay_0-64:{mountpoint:/var/lib/containers/storage/overlay/6f5c11500a059248b135ff7b80890d495eef09c060db0244e7a9d55a0f9365ee/merged major:0 minor:64 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/ce4333ee85e3c7dbdd0c79425360a8212bf078e4dfa7555fdb728516c048f9a3/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-71:{mountpoint:/var/lib/containers/storage/overlay/1f93c75e8f055c84e363107efa473bcebeedf351be1ba4ad55edec9d5114f722/merged major:0 minor:71 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/b626669e30496f79e6d107fad6d01a6f7525eff7ece9513541010d0f64d7cecb/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/c77163ed80c44955adffa5de7f73f6e7575d68d470e0ec47e01af9c62b4a481b/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/e94be990c1ee4bdf24cde7e4f9152d5517a26ab044c3c5872c2ac48d3b4918ae/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/4ccc4bba80066c8244a920025e10205beec19f23fb54d2d23b0aa645542c1197/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/01e6dadc033790627094a98dc9e08b7eebe8e63d9174a806468aab4a5aa8eb13/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-93:{mountpoint:/var/lib/containers/storage/overlay/95a7ae161603e162e1b75013e05cd4eaaa01288dc0b00ba97b1270d86afbd152/merged major:0 minor:93 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/63d9ccdc54b45e11efe7a4ae7bb55a134d256977050fa2f8244d9e8400c9931b/merged major:0 minor:96 fsType:overlay blockSize:0}] Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.057796 4859 manager.go:217] Machine: {Timestamp:2026-11-04 05:27:10.057274355 +0000 UTC m=+0.058910070 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099587 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2330219e9eb2fc31eabb1d3abbbfbd SystemUUID:ec233021-9e9e-b2fc-31ea-bb1d3abbbfbd BootID:9a27b69e-26d7-4df4-a738-76bc959f0dcd Filesystems:[{Device:/run/containers/storage/overlay-containers/c8bdb05cd3620d33090b4d863e9485be49c1e5fb7fc1bf8122b59451e19b15e4/userdata/shm DeviceMajor:0 DeviceMinor:69 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/901917c944bd1d7388e73ab52329720f1ebfb593f14f2276d89bff626841fcb2/userdata/shm DeviceMajor:0 DeviceMinor:208 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/c87f7dad-1181-4db5-b846-05d8fb0a33d5/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:211 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/11c2917eaf5a3eab504ea4d368cf93f8f34e45e865290a9be71acdda3333af46/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8f7d857f-84f9-4bde-9b5a-66164c90377e/volumes/kubernetes.io~projected/kube-api-access-hjmwm DeviceMajor:0 DeviceMinor:53 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/523aff5c-ac2b-43be-8e13-792f44b2c14e/volumes/kubernetes.io~projected/kube-api-access-c2flm DeviceMajor:0 DeviceMinor:203 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/d26b56ed-c6e3-4321-8044-d29e5fe081c8/volumes/kubernetes.io~projected/kube-api-access-xwj9c DeviceMajor:0 DeviceMinor:207 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-168 DeviceMajor:0 DeviceMinor:168 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-64 DeviceMajor:0 DeviceMinor:64 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d26b56ed-c6e3-4321-8044-d29e5fe081c8/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:201 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/48e160cc3b3d558da0739bcd3b1d4cbbaaec8d58904d767631cb7d25bf6d3f7d/userdata/shm DeviceMajor:0 DeviceMinor:227 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/26ae8a36-06bf-414e-b2bd-325033800b72/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:286 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/b25ddbfa-3e3e-49fd-870f-963d087f22a5/volumes/kubernetes.io~projected/kube-api-access-5rvs5 DeviceMajor:0 DeviceMinor:234 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8f7d857f-84f9-4bde-9b5a-66164c90377e/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:49 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-71 DeviceMajor:0 DeviceMinor:71 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/00a06b1e-791a-4652-823c-b73672b7e4b2/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:197 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/e10001da-2e9e-4b80-b6e0-0f1fca3df854/volumes/kubernetes.io~projected/kube-api-access-2fbtn DeviceMajor:0 DeviceMinor:202 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/00a06b1e-791a-4652-823c-b73672b7e4b2/volumes/kubernetes.io~projected/kube-api-access-ws4hd DeviceMajor:0 DeviceMinor:205 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/4a5449c3-4f21-44d2-9d10-9a0956aefdcf/volumes/kubernetes.io~projected/kube-api-access-fbx8t DeviceMajor:0 DeviceMinor:289 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/beca5780d1576f0b8b961482c174af7488c9be39a7136b68b4fad07dfa5673db/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/aa931f6e-a107-4ca3-bb54-e89a91d0f67a/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:220 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-292 DeviceMajor:0 DeviceMinor:292 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3f92655005734c04835a73a0a2c46997d2abe0b481a112c6395733a7731f21cd/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/aa931f6e-a107-4ca3-bb54-e89a91d0f67a/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:210 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6cf77b4d-32f5-42db-851f-3466ecf583c9/volumes/kubernetes.io~projected/kube-api-access-8d5fj DeviceMajor:0 DeviceMinor:55 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c65039c243a532bb0b419e22a11d20186c65571e11c590f0f61540e727812975/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/90fb287e-4af9-4e0b-919d-00713b37605c/volumes/kubernetes.io~projected/kube-api-access-m7488 DeviceMajor:0 DeviceMinor:233 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:212 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/5704adc4-5d8d-439e-ae45-2ab73b06bf14/volumes/kubernetes.io~projected/kube-api-access-vqqt4 DeviceMajor:0 DeviceMinor:237 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/38b751da-304f-4000-bf56-85e1d9aa8152/volumes/kubernetes.io~projected/kube-api-access-cpww4 DeviceMajor:0 DeviceMinor:59 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e44c5924347099be8c386a1c359936f6810d6c950da0b78944869ce8d5e90a1c/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb/volumes/kubernetes.io~projected/kube-api-access-x87gt DeviceMajor:0 DeviceMinor:57 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0036c1fcf22adf4a202a1b3d09c1f9e85b4fa8d40e0ff508318537d5859f6499/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/72dd8eb9-265d-4761-a507-d91e29606698/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:214 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3c81de14011f735b2a9889357de534ae6f3e9e5456a2fbf9a044be15cf4936c6/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-171 DeviceMajor:0 DeviceMinor:171 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8e46f3b116be4a32a514c6bd4dfe03c8f7a60ce85d4a42a1947d7652e289f25c/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/77f9e498-8338-4f89-ae13-e72e0108a602/volumes/kubernetes.io~projected/kube-api-access-2qw4k DeviceMajor:0 DeviceMinor:54 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/788d9f6ddbe47b4b03358ed63dc914669c3f40a5b8023bb26a91a0681452fddc/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/00a06b1e-791a-4652-823c-b73672b7e4b2/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:204 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/72dd8eb9-265d-4761-a507-d91e29606698/volumes/kubernetes.io~projected/kube-api-access-trzdq DeviceMajor:0 DeviceMinor:58 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/volumes/kubernetes.io~projected/kube-api-access-xjngp DeviceMajor:0 DeviceMinor:222 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/328415b5-7eda-429c-a74a-af29ba790b75/volumes/kubernetes.io~projected/kube-api-access-p4qpn DeviceMajor:0 DeviceMinor:306 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/03b38d7f8d93e469a723b47425c44e0922c67780ce06bb599cf5ee5089fd17ca/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/a10e4998-0b1d-4041-a022-9e36300936fa/volumes/kubernetes.io~projected/kube-api-access-n7x5w DeviceMajor:0 DeviceMinor:236 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/186c505a55d86fd094ca455e6d35af705ded27fd77d9cdabb1a3a4f9c7167e3d/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/76d053db0d1efc284602d29e6f33d8750bac470e4072bc672484db13f154f21c/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5e7c38cd-3943-484d-b433-cbec7bce35c6/volumes/kubernetes.io~projected/kube-api-access-qghpk DeviceMajor:0 DeviceMinor:56 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/00a06b1e-791a-4652-823c-b73672b7e4b2/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:45 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/c87f7dad-1181-4db5-b846-05d8fb0a33d5/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:215 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/191dfb71-401c-494a-b7bb-cbc6c3273430/volumes/kubernetes.io~projected/kube-api-access-gfptj DeviceMajor:0 DeviceMinor:223 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/90fb287e-4af9-4e0b-919d-00713b37605c/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:231 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-135 DeviceMajor:0 DeviceMinor:135 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2e2ba652-6141-4984-8384-51f9720a02ef/volumes/kubernetes.io~projected/kube-api-access-m6gpf DeviceMajor:0 DeviceMinor:206 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04/volumes/kubernetes.io~projected/kube-api-access-dtv2t DeviceMajor:0 DeviceMinor:225 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/aa931f6e-a107-4ca3-bb54-e89a91d0f67a/volumes/kubernetes.io~projected/kube-api-access-8prxh DeviceMajor:0 DeviceMinor:226 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/8f7d857f-84f9-4bde-9b5a-66164c90377e/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:50 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bf1d33ae64077954869a6356a46df86444a4af90371a39cab907c1f96f86f53e/userdata/shm DeviceMajor:0 DeviceMinor:218 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b45574024cbf01b45820b72330280a0aa370363457b75cfe34e03fd6361d4ee0/userdata/shm DeviceMajor:0 DeviceMinor:229 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-335 DeviceMajor:0 DeviceMinor:335 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/72dd8eb9-265d-4761-a507-d91e29606698/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:48 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/4a509700-2c8d-4113-9121-3ec287223f70/volumes/kubernetes.io~projected/kube-api-access-484lx DeviceMajor:0 DeviceMinor:51 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/aa931f6e-a107-4ca3-bb54-e89a91d0f67a/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:213 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/c87f7dad-1181-4db5-b846-05d8fb0a33d5/volumes/kubernetes.io~projected/kube-api-access-tz2hq DeviceMajor:0 DeviceMinor:221 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/981a3cfbef72505c1396f49c8f9064d2a5691eee3611d15ffbc3367c87ed44e7/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/428ba86e-0dd8-4a0a-b899-c7e9d568487c/volumes/kubernetes.io~projected/kube-api-access-9s7c8 DeviceMajor:0 DeviceMinor:52 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3266975edfbfb9106dd44dd9ed6ba2ba5a08329f9279468c1ca476ec9f277bf3/userdata/shm DeviceMajor:0 DeviceMinor:287 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/1a566754-7023-408d-bb6a-3eb984ffa7e3/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:47 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-122 DeviceMajor:0 DeviceMinor:122 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5704adc4-5d8d-439e-ae45-2ab73b06bf14/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:232 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/886edd3a-6e51-4ccf-92a8-9d8a3d447449/volumes/kubernetes.io~projected/kube-api-access-hbg2j DeviceMajor:0 DeviceMinor:235 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-93 DeviceMajor:0 DeviceMinor:93 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1291829ddcc5c473402ff9425d57f0601a0cb87b7d8769c787d41a3acdd96d64/userdata/shm DeviceMajor:0 DeviceMinor:290 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-319 DeviceMajor:0 DeviceMinor:319 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-324 DeviceMajor:0 DeviceMinor:324 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a70bed57c99af68bc7cf15bff43587864b448f233d985b7abf7a09f659f4162e/userdata/shm DeviceMajor:0 DeviceMinor:216 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/d998c080-e5b9-4de6-af9c-68a8c531257c/volumes/kubernetes.io~projected/kube-api-access-5p6g8 DeviceMajor:0 DeviceMinor:224 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/463c38893c854b23e9be888f730e7a64ecaba26981bc47571ba19fa0317bc9fa/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-279 DeviceMajor:0 DeviceMinor:279 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-315 DeviceMajor:0 DeviceMinor:315 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:11c2917eaf5a3ea MacAddress:f6:fc:a2:72:ca:37 Speed:10000 Mtu:8901} {Name:1291829ddcc5c47 MacAddress:6a:2a:e6:11:89:af Speed:10000 Mtu:8901} {Name:186c505a55d86fd MacAddress:36:f0:97:8a:6d:fd Speed:10000 Mtu:8901} {Name:3266975edfbfb91 MacAddress:ae:9a:fc:76:14:53 Speed:10000 Mtu:8901} {Name:3c81de14011f735 MacAddress:f6:c7:a8:7f:df:60 Speed:10000 Mtu:8901} {Name:48e160cc3b3d558 MacAddress:96:d1:46:48:27:99 Speed:10000 Mtu:8901} {Name:76d053db0d1efc2 MacAddress:66:6a:2b:c8:c4:26 Speed:10000 Mtu:8901} {Name:901917c944bd1d7 MacAddress:fa:ac:e8:28:3b:a5 Speed:10000 Mtu:8901} {Name:981a3cfbef72505 MacAddress:22:71:eb:c2:d7:50 Speed:10000 Mtu:8901} {Name:a70bed57c99af68 MacAddress:de:e4:e1:08:59:21 Speed:10000 Mtu:8901} {Name:b45574024cbf01b MacAddress:8e:45:60:86:eb:05 Speed:10000 Mtu:8901} {Name:bf1d33ae6407795 MacAddress:7e:74:dd:87:fe:e5 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:c2:64:f2:6b Speed:0 Mtu:9001} {Name:br-int MacAddress:2e:e1:4b:cc:18:7a Speed:0 Mtu:8901} {Name:ens5 MacAddress:06:ff:c2:64:f2:6b Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:b2:65:ad:d9:ef:de Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:de:40:35:b1:3c:8c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.058114 4859 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.058154 4859 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.058419 4859 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.058438 4859 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-143.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.058557 4859 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.058565 4859 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.058575 4859 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.058594 4859 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.058611 4859 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.058784 4859 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.058883 4859 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.058925 4859 kubelet.go:493] "Attempting to sync node with API server" Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.058934 4859 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.058944 4859 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.058952 4859 kubelet.go:405] "Adding apiserver pod source" Nov 04 05:27:10.060318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.058960 4859 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 05:27:10.061921 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.061901 4859 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 05:27:10.061921 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.061920 4859 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 05:27:10.062353 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.062335 4859 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 05:27:10.064414 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064401 4859 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 05:27:10.064414 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064414 4859 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 05:27:10.064509 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064500 4859 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Nov 04 05:27:10.064703 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064692 4859 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 05:27:10.064736 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064721 4859 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 05:27:10.064822 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064813 4859 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 05:27:10.064859 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064826 4859 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 05:27:10.064859 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064833 4859 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 05:27:10.064859 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064838 4859 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 05:27:10.064859 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064844 4859 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 05:27:10.064859 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064850 4859 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 05:27:10.064859 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064856 4859 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 05:27:10.065010 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064862 4859 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 05:27:10.065010 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064869 4859 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 05:27:10.065010 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064875 4859 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 05:27:10.065010 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064888 4859 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 05:27:10.065010 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064898 4859 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 05:27:10.065010 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064919 4859 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 05:27:10.065010 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.064924 4859 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 05:27:10.068235 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.068223 4859 server.go:1263] "Started kubelet" Nov 04 05:27:10.068473 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.068310 4859 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 05:27:10.069119 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.068331 4859 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 05:27:10.069119 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.068647 4859 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 05:27:10.069119 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.069026 4859 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 05:27:10.069487 ip-10-0-133-143 systemd[1]: Started Kubernetes Kubelet. Nov 04 05:27:10.070030 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.070013 4859 server.go:317] "Adding debug handlers to kubelet server" Nov 04 05:27:10.073222 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.072977 4859 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 05:27:10.078109 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.074097 4859 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 05:27:10.078109 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.074170 4859 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 05:21:28 +0000 UTC" deadline="2028-05-10 03:30:54.078791793 +0000 UTC" Nov 04 05:27:10.078109 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.074206 4859 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13270h3m44.004589584s" Nov 04 05:27:10.078109 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.074372 4859 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 05:27:10.078109 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.074388 4859 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 05:27:10.078109 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:10.074387 4859 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-143.us-west-2.compute.internal\" not found" Nov 04 05:27:10.078109 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.074507 4859 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 05:27:10.078109 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.075813 4859 factory.go:55] Registering systemd factory Nov 04 05:27:10.078109 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.075830 4859 factory.go:223] Registration of the systemd container factory successfully Nov 04 05:27:10.078109 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.076078 4859 factory.go:153] Registering CRI-O factory Nov 04 05:27:10.078109 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.076094 4859 factory.go:223] Registration of the crio container factory successfully Nov 04 05:27:10.078109 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.076173 4859 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 05:27:10.078109 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.076200 4859 factory.go:103] Registering Raw factory Nov 04 05:27:10.078109 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.076215 4859 manager.go:1199] Started watching for new ooms in manager Nov 04 05:27:10.078109 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.076602 4859 manager.go:319] Starting recovery of all containers Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.079990 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa931f6e-a107-4ca3-bb54-e89a91d0f67a" volumeName="kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-kube-api-access-8prxh" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080024 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04" volumeName="kubernetes.io/configmap/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-config" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080038 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00a06b1e-791a-4652-823c-b73672b7e4b2" volumeName="kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-bound-sa-token" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080050 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2e2ba652-6141-4984-8384-51f9720a02ef" volumeName="kubernetes.io/projected/2e2ba652-6141-4984-8384-51f9720a02ef-kube-api-access-m6gpf" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080062 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b25ddbfa-3e3e-49fd-870f-963d087f22a5" volumeName="kubernetes.io/projected/b25ddbfa-3e3e-49fd-870f-963d087f22a5-kube-api-access-5rvs5" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080075 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04" volumeName="kubernetes.io/projected/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-kube-api-access-dtv2t" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080086 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d26b56ed-c6e3-4321-8044-d29e5fe081c8" volumeName="kubernetes.io/configmap/d26b56ed-c6e3-4321-8044-d29e5fe081c8-config" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080098 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef5a3412-cd67-424f-bb1f-1ec122d5e3eb" volumeName="kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cni-sysctl-allowlist" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080112 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef5a3412-cd67-424f-bb1f-1ec122d5e3eb" volumeName="kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cni-binary-copy" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080123 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5e7c38cd-3943-484d-b433-cbec7bce35c6" volumeName="kubernetes.io/configmap/5e7c38cd-3943-484d-b433-cbec7bce35c6-serviceca" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080134 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77f9e498-8338-4f89-ae13-e72e0108a602" volumeName="kubernetes.io/configmap/77f9e498-8338-4f89-ae13-e72e0108a602-cni-binary-copy" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080150 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77f9e498-8338-4f89-ae13-e72e0108a602" volumeName="kubernetes.io/projected/77f9e498-8338-4f89-ae13-e72e0108a602-kube-api-access-2qw4k" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080162 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f7d857f-84f9-4bde-9b5a-66164c90377e" volumeName="kubernetes.io/empty-dir/8f7d857f-84f9-4bde-9b5a-66164c90377e-tmp" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080173 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef5a3412-cd67-424f-bb1f-1ec122d5e3eb" volumeName="kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-whereabouts-flatfile-configmap" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080187 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="428ba86e-0dd8-4a0a-b899-c7e9d568487c" volumeName="kubernetes.io/empty-dir/428ba86e-0dd8-4a0a-b899-c7e9d568487c-tmp-dir" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080223 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5704adc4-5d8d-439e-ae45-2ab73b06bf14" volumeName="kubernetes.io/projected/5704adc4-5d8d-439e-ae45-2ab73b06bf14-kube-api-access-vqqt4" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080234 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="886edd3a-6e51-4ccf-92a8-9d8a3d447449" volumeName="kubernetes.io/configmap/886edd3a-6e51-4ccf-92a8-9d8a3d447449-config-volume" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080247 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f7d857f-84f9-4bde-9b5a-66164c90377e" volumeName="kubernetes.io/projected/8f7d857f-84f9-4bde-9b5a-66164c90377e-kube-api-access-hjmwm" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080259 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90fb287e-4af9-4e0b-919d-00713b37605c" volumeName="kubernetes.io/empty-dir/90fb287e-4af9-4e0b-919d-00713b37605c-tmp" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080277 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90fb287e-4af9-4e0b-919d-00713b37605c" volumeName="kubernetes.io/empty-dir/90fb287e-4af9-4e0b-919d-00713b37605c-snapshots" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080288 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90fb287e-4af9-4e0b-919d-00713b37605c" volumeName="kubernetes.io/projected/90fb287e-4af9-4e0b-919d-00713b37605c-kube-api-access-m7488" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080299 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90fb287e-4af9-4e0b-919d-00713b37605c" volumeName="kubernetes.io/secret/90fb287e-4af9-4e0b-919d-00713b37605c-serving-cert" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080310 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5704adc4-5d8d-439e-ae45-2ab73b06bf14" volumeName="kubernetes.io/secret/5704adc4-5d8d-439e-ae45-2ab73b06bf14-cert" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080323 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6cf77b4d-32f5-42db-851f-3466ecf583c9" volumeName="kubernetes.io/projected/6cf77b4d-32f5-42db-851f-3466ecf583c9-kube-api-access-8d5fj" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080335 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72dd8eb9-265d-4761-a507-d91e29606698" volumeName="kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-ovnkube-script-lib" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080347 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a35474ea-79b9-4d84-81b1-39db689685d2" volumeName="kubernetes.io/configmap/a35474ea-79b9-4d84-81b1-39db689685d2-nginx-conf" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080359 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa931f6e-a107-4ca3-bb54-e89a91d0f67a" volumeName="kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-trusted-ca" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080371 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d26b56ed-c6e3-4321-8044-d29e5fe081c8" volumeName="kubernetes.io/projected/d26b56ed-c6e3-4321-8044-d29e5fe081c8-kube-api-access-xwj9c" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080388 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d998c080-e5b9-4de6-af9c-68a8c531257c" volumeName="kubernetes.io/projected/d998c080-e5b9-4de6-af9c-68a8c531257c-kube-api-access-5p6g8" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080402 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00a06b1e-791a-4652-823c-b73672b7e4b2" volumeName="kubernetes.io/empty-dir/00a06b1e-791a-4652-823c-b73672b7e4b2-ca-trust-extracted" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080414 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00a06b1e-791a-4652-823c-b73672b7e4b2" volumeName="kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-kube-api-access-ws4hd" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080441 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a5449c3-4f21-44d2-9d10-9a0956aefdcf" volumeName="kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080454 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00a06b1e-791a-4652-823c-b73672b7e4b2" volumeName="kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-trusted-ca" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080467 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00a06b1e-791a-4652-823c-b73672b7e4b2" volumeName="kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-installation-pull-secrets" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080479 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="58bf7d20-81e0-4dbc-85f2-32ded8f14e5f" volumeName="kubernetes.io/configmap/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-trusted-ca" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080491 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72dd8eb9-265d-4761-a507-d91e29606698" volumeName="kubernetes.io/projected/72dd8eb9-265d-4761-a507-d91e29606698-kube-api-access-trzdq" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080504 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a10e4998-0b1d-4041-a022-9e36300936fa" volumeName="kubernetes.io/projected/a10e4998-0b1d-4041-a022-9e36300936fa-kube-api-access-n7x5w" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080518 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="191dfb71-401c-494a-b7bb-cbc6c3273430" volumeName="kubernetes.io/configmap/191dfb71-401c-494a-b7bb-cbc6c3273430-telemetry-config" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080531 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="58bf7d20-81e0-4dbc-85f2-32ded8f14e5f" volumeName="kubernetes.io/projected/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-kube-api-access-xjngp" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080556 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72dd8eb9-265d-4761-a507-d91e29606698" volumeName="kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-ovnkube-config" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080569 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90fb287e-4af9-4e0b-919d-00713b37605c" volumeName="kubernetes.io/configmap/90fb287e-4af9-4e0b-919d-00713b37605c-trusted-ca-bundle" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080582 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c87f7dad-1181-4db5-b846-05d8fb0a33d5" volumeName="kubernetes.io/projected/c87f7dad-1181-4db5-b846-05d8fb0a33d5-kube-api-access-tz2hq" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080594 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00a06b1e-791a-4652-823c-b73672b7e4b2" volumeName="kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-certificates" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080607 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="328415b5-7eda-429c-a74a-af29ba790b75" volumeName="kubernetes.io/projected/328415b5-7eda-429c-a74a-af29ba790b75-kube-api-access-p4qpn" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080619 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a509700-2c8d-4113-9121-3ec287223f70" volumeName="kubernetes.io/projected/4a509700-2c8d-4113-9121-3ec287223f70-kube-api-access-484lx" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080631 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72dd8eb9-265d-4761-a507-d91e29606698" volumeName="kubernetes.io/secret/72dd8eb9-265d-4761-a507-d91e29606698-ovn-node-metrics-cert" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080641 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="886edd3a-6e51-4ccf-92a8-9d8a3d447449" volumeName="kubernetes.io/empty-dir/886edd3a-6e51-4ccf-92a8-9d8a3d447449-tmp-dir" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080655 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f7d857f-84f9-4bde-9b5a-66164c90377e" volumeName="kubernetes.io/empty-dir/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-tuned" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080682 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90fb287e-4af9-4e0b-919d-00713b37605c" volumeName="kubernetes.io/configmap/90fb287e-4af9-4e0b-919d-00713b37605c-service-ca-bundle" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080693 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c87f7dad-1181-4db5-b846-05d8fb0a33d5" volumeName="kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-default-certificate" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080715 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6cf77b4d-32f5-42db-851f-3466ecf583c9" volumeName="kubernetes.io/configmap/6cf77b4d-32f5-42db-851f-3466ecf583c9-iptables-alerter-script" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080722 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa931f6e-a107-4ca3-bb54-e89a91d0f67a" volumeName="kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-certificates" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080729 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a566754-7023-408d-bb6a-3eb984ffa7e3" volumeName="kubernetes.io/configmap/1a566754-7023-408d-bb6a-3eb984ffa7e3-konnectivity-ca" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080738 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a566754-7023-408d-bb6a-3eb984ffa7e3" volumeName="kubernetes.io/secret/1a566754-7023-408d-bb6a-3eb984ffa7e3-agent-certs" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080746 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa931f6e-a107-4ca3-bb54-e89a91d0f67a" volumeName="kubernetes.io/empty-dir/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-ca-trust-extracted" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080756 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00a06b1e-791a-4652-823c-b73672b7e4b2" volumeName="kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-image-registry-private-configuration" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080787 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="38b751da-304f-4000-bf56-85e1d9aa8152" volumeName="kubernetes.io/projected/38b751da-304f-4000-bf56-85e1d9aa8152-kube-api-access-cpww4" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080797 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa931f6e-a107-4ca3-bb54-e89a91d0f67a" volumeName="kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-bound-sa-token" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080808 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d998c080-e5b9-4de6-af9c-68a8c531257c" volumeName="kubernetes.io/configmap/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-config" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080818 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="58bf7d20-81e0-4dbc-85f2-32ded8f14e5f" volumeName="kubernetes.io/secret/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-serving-cert" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080825 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26ae8a36-06bf-414e-b2bd-325033800b72" volumeName="kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080832 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="523aff5c-ac2b-43be-8e13-792f44b2c14e" volumeName="kubernetes.io/projected/523aff5c-ac2b-43be-8e13-792f44b2c14e-kube-api-access-c2flm" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080841 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="58bf7d20-81e0-4dbc-85f2-32ded8f14e5f" volumeName="kubernetes.io/configmap/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-config" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080851 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5e7c38cd-3943-484d-b433-cbec7bce35c6" volumeName="kubernetes.io/projected/5e7c38cd-3943-484d-b433-cbec7bce35c6-kube-api-access-qghpk" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080858 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d26b56ed-c6e3-4321-8044-d29e5fe081c8" volumeName="kubernetes.io/secret/d26b56ed-c6e3-4321-8044-d29e5fe081c8-serving-cert" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080867 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72dd8eb9-265d-4761-a507-d91e29606698" volumeName="kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-env-overrides" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080875 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77f9e498-8338-4f89-ae13-e72e0108a602" volumeName="kubernetes.io/configmap/77f9e498-8338-4f89-ae13-e72e0108a602-multus-daemon-config" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080882 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa931f6e-a107-4ca3-bb54-e89a91d0f67a" volumeName="kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-installation-pull-secrets" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080890 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa931f6e-a107-4ca3-bb54-e89a91d0f67a" volumeName="kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-image-registry-private-configuration" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080897 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c87f7dad-1181-4db5-b846-05d8fb0a33d5" volumeName="kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-stats-auth" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080904 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef5a3412-cd67-424f-bb1f-1ec122d5e3eb" volumeName="kubernetes.io/projected/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-kube-api-access-x87gt" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080911 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="428ba86e-0dd8-4a0a-b899-c7e9d568487c" volumeName="kubernetes.io/projected/428ba86e-0dd8-4a0a-b899-c7e9d568487c-kube-api-access-9s7c8" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080919 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="886edd3a-6e51-4ccf-92a8-9d8a3d447449" volumeName="kubernetes.io/projected/886edd3a-6e51-4ccf-92a8-9d8a3d447449-kube-api-access-hbg2j" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080927 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04" volumeName="kubernetes.io/secret/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-serving-cert" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080934 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e10001da-2e9e-4b80-b6e0-0f1fca3df854" volumeName="kubernetes.io/projected/e10001da-2e9e-4b80-b6e0-0f1fca3df854-kube-api-access-2fbtn" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080941 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="191dfb71-401c-494a-b7bb-cbc6c3273430" volumeName="kubernetes.io/projected/191dfb71-401c-494a-b7bb-cbc6c3273430-kube-api-access-gfptj" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080948 4859 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="328415b5-7eda-429c-a74a-af29ba790b75" volumeName="kubernetes.io/empty-dir/328415b5-7eda-429c-a74a-af29ba790b75-data-volume" seLinuxMountContext="" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080955 4859 reconstruct.go:97] "Volume reconstruction finished" Nov 04 05:27:10.081180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.080961 4859 reconciler.go:29] "Reconciler: start to sync state" Nov 04 05:27:10.087772 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.087755 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/1.log" Nov 04 05:27:10.091893 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.091864 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/1.log" Nov 04 05:27:10.093734 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.093708 4859 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 05:27:10.108116 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.108096 4859 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 05:27:10.108116 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.108117 4859 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 05:27:10.108236 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.108133 4859 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 05:27:10.108236 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:10.108173 4859 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Nov 04 05:27:10.114968 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.114943 4859 generic.go:356] "Generic (PLEG): container finished" podID="94f347668688382ceddf8a0c03be4355" containerID="0e9b5e18139c670f433c1cf7029708506efb8e28e9f271d6942a649862096747" exitCode=0 Nov 04 05:27:10.132234 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.132201 4859 generic.go:356] "Generic (PLEG): container finished" podID="ef5a3412-cd67-424f-bb1f-1ec122d5e3eb" containerID="f97fd050565f600b43a37a265bce54feb2e241b2f75663ba5c6b30ece348fce1" exitCode=0 Nov 04 05:27:10.132234 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.132233 4859 generic.go:356] "Generic (PLEG): container finished" podID="ef5a3412-cd67-424f-bb1f-1ec122d5e3eb" containerID="3083708d8f0d881f974705f48071193cc43c964153052ec1212abf95cc0d8a4e" exitCode=0 Nov 04 05:27:10.132399 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.132250 4859 generic.go:356] "Generic (PLEG): container finished" podID="ef5a3412-cd67-424f-bb1f-1ec122d5e3eb" containerID="ee6bdc76eeb9716dc5a327d5d328c4bd6ba78a73189e2c7be25e8fb314ed39bb" exitCode=0 Nov 04 05:27:10.132399 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.132268 4859 generic.go:356] "Generic (PLEG): container finished" podID="ef5a3412-cd67-424f-bb1f-1ec122d5e3eb" containerID="237e08585ba6eed1150f0ab2042140d946d4b7d49218a505161a8267b1728265" exitCode=0 Nov 04 05:27:10.132399 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.132278 4859 generic.go:356] "Generic (PLEG): container finished" podID="ef5a3412-cd67-424f-bb1f-1ec122d5e3eb" containerID="3d7a67791b86e74c89f18c36b21dcb03dd50729fc1c8139b18ea87c1fcb158c6" exitCode=0 Nov 04 05:27:10.132399 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.132287 4859 generic.go:356] "Generic (PLEG): container finished" podID="ef5a3412-cd67-424f-bb1f-1ec122d5e3eb" containerID="b150a84ce212f377826faaa957ce27d0a1d16422427b3db903dcb1bc94f7ef9e" exitCode=0 Nov 04 05:27:10.147060 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.147040 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/1.log" Nov 04 05:27:10.147149 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.147070 4859 generic.go:356] "Generic (PLEG): container finished" podID="58bf7d20-81e0-4dbc-85f2-32ded8f14e5f" containerID="28879b3159ff2859835187b11135533fdeb2de72dd6adcc016e7382ffad5bf8b" exitCode=255 Nov 04 05:27:10.165655 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.165635 4859 manager.go:324] Recovery completed Nov 04 05:27:10.174619 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:10.174597 4859 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-143.us-west-2.compute.internal\" not found" Nov 04 05:27:10.181702 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.181688 4859 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:10.182700 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.182684 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:10.182791 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.182716 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:10.182791 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.182732 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:10.185305 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.185286 4859 cpu_manager.go:226] "Starting" policy="none" Nov 04 05:27:10.185305 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.185301 4859 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 05:27:10.185413 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.185302 4859 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:10.185413 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.185326 4859 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 05:27:10.185480 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.185463 4859 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Nov 04 05:27:10.185521 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.185482 4859 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Nov 04 05:27:10.185521 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.185508 4859 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Nov 04 05:27:10.185521 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.185513 4859 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Nov 04 05:27:10.185521 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.185518 4859 policy_none.go:50] "Start" Nov 04 05:27:10.185625 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.185527 4859 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 05:27:10.185625 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.185537 4859 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 05:27:10.185715 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.185704 4859 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Nov 04 05:27:10.185715 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.185715 4859 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Nov 04 05:27:10.185778 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.185720 4859 policy_none.go:44] "Start" Nov 04 05:27:10.188284 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.188254 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:10.188356 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.188314 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:10.188356 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.188329 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:10.191227 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.191203 4859 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 05:27:10.194658 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.194640 4859 manager.go:341] "Starting Device Plugin manager" Nov 04 05:27:10.194753 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:10.194674 4859 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 05:27:10.194753 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.194686 4859 server.go:87] "Starting device plugin registration server" Nov 04 05:27:10.194996 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.194983 4859 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 05:27:10.195028 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.194998 4859 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 05:27:10.195104 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.195085 4859 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 05:27:10.195217 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.195185 4859 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 05:27:10.195297 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.195226 4859 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 05:27:10.195297 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.195233 4859 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 05:27:10.195469 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.195321 4859 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T05:27:10.195215458Z","UUID":"e2fd1284-952f-4e26-8302-3e41d4273b89","Handler":null,"Name":"","Endpoint":""} Nov 04 05:27:10.196642 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.196620 4859 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 05:27:10.196733 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.196649 4859 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 05:27:10.196875 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:10.196856 4859 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 05:27:10.196965 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:10.196895 4859 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-143.us-west-2.compute.internal\" not found" Nov 04 05:27:10.209304 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.208839 4859 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal"] Nov 04 05:27:10.209385 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.209373 4859 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:10.210135 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.210120 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:10.210209 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.210150 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:10.210209 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.210163 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:10.210425 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.210415 4859 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:10.210553 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.210540 4859 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:10.211116 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.211100 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:10.211176 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.211132 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:10.211176 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.211143 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:10.211176 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.211104 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:10.211176 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.211174 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:10.211312 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.211186 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:10.211312 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:10.211303 4859 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-143.us-west-2.compute.internal\" not found" node="ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.211387 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.211363 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" event={"ID":"9d7645ed4719bb9894aec65f339b9ca6","Type":"ContainerStarted","Data":"9cd00cc4a3f6d8dd6a55c75aa41137f2b722505b0fe10530c350bde54f4604cc"} Nov 04 05:27:10.211417 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.211394 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" event={"ID":"9d7645ed4719bb9894aec65f339b9ca6","Type":"ContainerStarted","Data":"463c38893c854b23e9be888f730e7a64ecaba26981bc47571ba19fa0317bc9fa"} Nov 04 05:27:10.211417 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.211407 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" event={"ID":"94f347668688382ceddf8a0c03be4355","Type":"ContainerStarted","Data":"d02cc59c406666fc0b1553ca8a0c89808e3a58e8a347fdc7ea7d754928c04d63"} Nov 04 05:27:10.211473 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.211419 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" event={"ID":"94f347668688382ceddf8a0c03be4355","Type":"ContainerDied","Data":"0e9b5e18139c670f433c1cf7029708506efb8e28e9f271d6942a649862096747"} Nov 04 05:27:10.211473 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.211432 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" event={"ID":"94f347668688382ceddf8a0c03be4355","Type":"ContainerStarted","Data":"e44c5924347099be8c386a1c359936f6810d6c950da0b78944869ce8d5e90a1c"} Nov 04 05:27:10.211525 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.211482 4859 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:10.212104 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.212089 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:10.212180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.212116 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:10.212180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.212126 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:10.212254 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:10.212233 4859 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-143.us-west-2.compute.internal\" not found" node="ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.271768 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.271746 4859 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:27:10.271768 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.271752 4859 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:27:10.271880 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.271787 4859 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Nov 04 05:27:10.272421 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.272402 4859 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:27:10.276611 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.276593 4859 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.295751 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.295730 4859 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:10.296493 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.296477 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:10.296570 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.296510 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:10.296570 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.296525 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:10.296683 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.296577 4859 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.322553 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.322528 4859 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:27:10.363001 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.362978 4859 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:27:10.363079 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:10.363018 4859 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.363079 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.363033 4859 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.372245 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.372227 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94f347668688382ceddf8a0c03be4355-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"94f347668688382ceddf8a0c03be4355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.372291 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.372252 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9d7645ed4719bb9894aec65f339b9ca6-config\") pod \"kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"9d7645ed4719bb9894aec65f339b9ca6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.372291 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.372277 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94f347668688382ceddf8a0c03be4355-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"94f347668688382ceddf8a0c03be4355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.415295 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.415274 4859 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.415360 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.415325 4859 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.415791 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.415778 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeNotReady" Nov 04 05:27:10.415864 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.415808 4859 setters.go:546] "Node became not ready" node="ip-10-0-133-143.us-west-2.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-04T05:27:10Z","lastTransitionTime":"2026-11-04T05:27:10Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"} Nov 04 05:27:10.437847 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.437826 4859 nodeinfomanager.go:471] existing CSINode "ip-10-0-133-143.us-west-2.compute.internal" is owned by different node (oldNodeID="1f4081a8-a639-419f-b462-125241195703", newNodeID=""), cleaning up... Nov 04 05:27:10.438422 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.438408 4859 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:27:10.438487 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:10.438444 4859 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.472860 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.472811 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9d7645ed4719bb9894aec65f339b9ca6-config\") pod \"kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"9d7645ed4719bb9894aec65f339b9ca6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.472860 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.472835 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94f347668688382ceddf8a0c03be4355-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"94f347668688382ceddf8a0c03be4355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.472860 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.472853 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94f347668688382ceddf8a0c03be4355-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"94f347668688382ceddf8a0c03be4355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.472989 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.472896 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94f347668688382ceddf8a0c03be4355-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"94f347668688382ceddf8a0c03be4355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.472989 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.472923 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9d7645ed4719bb9894aec65f339b9ca6-config\") pod \"kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"9d7645ed4719bb9894aec65f339b9ca6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:10.472989 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:10.472923 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94f347668688382ceddf8a0c03be4355-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal\" (UID: \"94f347668688382ceddf8a0c03be4355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:11.059908 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.059876 4859 apiserver.go:52] "Watching apiserver" Nov 04 05:27:11.162132 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.161549 4859 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:11.162290 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.162120 4859 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:11.214000 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.213969 4859 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 05:27:11.214641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.214610 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-wrd6s","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal","openshift-network-operator/iptables-alerter-q7mtr","openshift-network-diagnostics/network-check-source-6f78d59f99-xk9lw","kube-system/konnectivity-agent-x27vz","openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg","openshift-multus/multus-additional-cni-plugins-ddzfn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn","kube-system/global-pull-secret-syncer-nk2gk","openshift-ingress-canary/ingress-canary-767sh","openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9","openshift-console-operator/console-operator-7cd56b8dbc-qnfh8","openshift-ingress/router-default-58c967f969-4xztc","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl","kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal","openshift-dns/node-resolver-87f78","openshift-image-registry/image-registry-6469786477-m4mgb","openshift-multus/network-metrics-daemon-55qcs","openshift-ovn-kubernetes/ovnkube-node-7tqkj","openshift-dns/dns-default-brcbz","kube-system/kas-connection-checker-685895c7c9-h5ccq","kube-system/kas-connection-checker-685895c7c9-pw5zp","kube-system/kas-connection-checker-685895c7c9-5zj4b","openshift-network-diagnostics/network-check-target-27hf6","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j8lxw","openshift-multus/multus-vmgxh","openshift-insights/insights-runtime-extractor-xfgjz","openshift-image-registry/node-ca-m9v8n","openshift-insights/insights-operator-7c4986d449-b24ft","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl","openshift-image-registry/image-registry-8f7d8c7c6-q48ph","openshift-network-console/networking-console-plugin-86c478bb88-pgdqn"] Nov 04 05:27:11.215923 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.215905 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:11.216505 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.216489 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:27:11.216823 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.216808 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:11.218826 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.218810 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.219205 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.219189 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:27:11.221094 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.221078 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:27:11.223417 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.223403 4859 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:27:11.223480 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.223439 4859 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:11.223848 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.223835 4859 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:27:11.223890 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.223874 4859 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-143.us-west-2.compute.internal" Nov 04 05:27:11.225614 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.225596 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.229581 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.229555 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:11.230415 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.230391 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:11.270296 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.270271 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.270985 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.270964 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Nov 04 05:27:11.270985 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.270983 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Nov 04 05:27:11.271157 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.271006 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Nov 04 05:27:11.271157 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.270969 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Nov 04 05:27:11.271157 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.271029 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Nov 04 05:27:11.271157 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.271036 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Nov 04 05:27:11.271157 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.271035 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Nov 04 05:27:11.271157 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.271054 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Nov 04 05:27:11.271157 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.271059 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Nov 04 05:27:11.271157 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.271078 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Nov 04 05:27:11.271157 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.271032 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Nov 04 05:27:11.271437 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.271242 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.275097 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.275081 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 05:27:11.275219 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.275173 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-8r47x\"" Nov 04 05:27:11.275397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.275382 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hqws2\"" Nov 04 05:27:11.275486 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.275428 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Nov 04 05:27:11.275557 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.275530 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r57nx\"" Nov 04 05:27:11.275716 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.275704 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Nov 04 05:27:11.276729 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.276716 4859 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 05:27:11.278891 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.278874 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-systemd\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.278929 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.278903 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90fb287e-4af9-4e0b-919d-00713b37605c-tmp\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:27:11.278929 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.278920 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-systemd\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.279006 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.278936 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:27:11.279006 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.278994 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qghpk\" (UniqueName: \"kubernetes.io/projected/5e7c38cd-3943-484d-b433-cbec7bce35c6-kube-api-access-qghpk\") pod \"node-ca-m9v8n\" (UID: \"5e7c38cd-3943-484d-b433-cbec7bce35c6\") " pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:27:11.279100 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279018 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.279100 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279070 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90fb287e-4af9-4e0b-919d-00713b37605c-tmp\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:27:11.279194 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279120 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-ca-trust-extracted\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.279194 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279141 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8prxh\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-kube-api-access-8prxh\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.279194 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279160 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:11.279194 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279183 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d26b56ed-c6e3-4321-8044-d29e5fe081c8-config\") pod \"service-ca-operator-544d749c8c-kqsd9\" (UID: \"d26b56ed-c6e3-4321-8044-d29e5fe081c8\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:27:11.279361 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279216 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-ca-trust-extracted\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.279361 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279225 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysctl-conf\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.279361 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279252 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-trusted-ca\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.279361 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279297 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7x5w\" (UniqueName: \"kubernetes.io/projected/a10e4998-0b1d-4041-a022-9e36300936fa-kube-api-access-n7x5w\") pod \"volume-data-source-validator-58c9f5f46d-j8lxw\" (UID: \"a10e4998-0b1d-4041-a022-9e36300936fa\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j8lxw" Nov 04 05:27:11.279539 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279361 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cnibin\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.279539 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279378 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.279539 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279396 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a566754-7023-408d-bb6a-3eb984ffa7e3-agent-certs\") pod \"konnectivity-agent-x27vz\" (UID: \"1a566754-7023-408d-bb6a-3eb984ffa7e3\") " pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:27:11.279539 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279485 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.279539 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279517 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/00a06b1e-791a-4652-823c-b73672b7e4b2-ca-trust-extracted\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.279738 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279544 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-etc-selinux\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.279738 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279602 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/00a06b1e-791a-4652-823c-b73672b7e4b2-ca-trust-extracted\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.279738 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279621 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-run-ovn-kubernetes\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.279738 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279647 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-cni-netd\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.279738 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279675 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.279738 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279705 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-socket-dir-parent\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.279738 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279733 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6gpf\" (UniqueName: \"kubernetes.io/projected/2e2ba652-6141-4984-8384-51f9720a02ef-kube-api-access-m6gpf\") pod \"kas-connection-checker-685895c7c9-pw5zp\" (UID: \"2e2ba652-6141-4984-8384-51f9720a02ef\") " pod="kube-system/kas-connection-checker-685895c7c9-pw5zp" Nov 04 05:27:11.279933 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279764 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-lib-modules\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.279933 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279850 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-log-socket\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.279933 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279867 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-serving-cert\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:27:11.279933 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279913 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysconfig\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.280040 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279941 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/886edd3a-6e51-4ccf-92a8-9d8a3d447449-tmp-dir\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:11.280040 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.279974 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/90fb287e-4af9-4e0b-919d-00713b37605c-snapshots\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:27:11.280040 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280027 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/886edd3a-6e51-4ccf-92a8-9d8a3d447449-tmp-dir\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:11.280040 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280032 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-serving-cert\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:27:11.280149 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280048 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/90fb287e-4af9-4e0b-919d-00713b37605c-snapshots\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:27:11.280149 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280054 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77f9e498-8338-4f89-ae13-e72e0108a602-multus-daemon-config\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.280149 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280074 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysctl-d\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.280149 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280107 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-ovnkube-config\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.280149 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280127 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-env-overrides\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.280149 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280143 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjmwm\" (UniqueName: \"kubernetes.io/projected/8f7d857f-84f9-4bde-9b5a-66164c90377e-kube-api-access-hjmwm\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.280371 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280160 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-29hbl\" (UID: \"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:27:11.280371 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280186 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-socket-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.280371 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280213 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-run-netns\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.280371 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280238 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-certificates\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.280371 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280267 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-kubelet\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.280371 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280299 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s7c8\" (UniqueName: \"kubernetes.io/projected/428ba86e-0dd8-4a0a-b899-c7e9d568487c-kube-api-access-9s7c8\") pod \"node-resolver-87f78\" (UID: \"428ba86e-0dd8-4a0a-b899-c7e9d568487c\") " pod="openshift-dns/node-resolver-87f78" Nov 04 05:27:11.280371 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280319 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p6g8\" (UniqueName: \"kubernetes.io/projected/d998c080-e5b9-4de6-af9c-68a8c531257c-kube-api-access-5p6g8\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:11.280371 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280335 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a566754-7023-408d-bb6a-3eb984ffa7e3-konnectivity-ca\") pod \"konnectivity-agent-x27vz\" (UID: \"1a566754-7023-408d-bb6a-3eb984ffa7e3\") " pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:27:11.280371 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280351 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/886edd3a-6e51-4ccf-92a8-9d8a3d447449-config-volume\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:11.280371 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280367 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90fb287e-4af9-4e0b-919d-00713b37605c-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:27:11.280820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280392 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-installation-pull-secrets\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.280820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280417 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/328415b5-7eda-429c-a74a-af29ba790b75-crio-socket\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:11.280820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280476 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-stats-auth\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:11.280820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280500 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:11.280820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280527 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6cf77b4d-32f5-42db-851f-3466ecf583c9-iptables-alerter-script\") pod \"iptables-alerter-q7mtr\" (UID: \"6cf77b4d-32f5-42db-851f-3466ecf583c9\") " pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:27:11.280820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280552 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-conf-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.280820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280578 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:11.280820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280605 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws4hd\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-kube-api-access-ws4hd\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.280820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280634 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-installation-pull-secrets\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.280820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280654 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-484lx\" (UniqueName: \"kubernetes.io/projected/4a509700-2c8d-4113-9121-3ec287223f70-kube-api-access-484lx\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:27:11.280820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280709 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-ovn\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.280820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280737 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-trusted-ca\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:27:11.280820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280712 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-stats-auth\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:11.280820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280776 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d26b56ed-c6e3-4321-8044-d29e5fe081c8-serving-cert\") pod \"service-ca-operator-544d749c8c-kqsd9\" (UID: \"d26b56ed-c6e3-4321-8044-d29e5fe081c8\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:27:11.280820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280808 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwj9c\" (UniqueName: \"kubernetes.io/projected/d26b56ed-c6e3-4321-8044-d29e5fe081c8-kube-api-access-xwj9c\") pod \"service-ca-operator-544d749c8c-kqsd9\" (UID: \"d26b56ed-c6e3-4321-8044-d29e5fe081c8\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280834 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/428ba86e-0dd8-4a0a-b899-c7e9d568487c-hosts-file\") pod \"node-resolver-87f78\" (UID: \"428ba86e-0dd8-4a0a-b899-c7e9d568487c\") " pod="openshift-dns/node-resolver-87f78" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280852 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-image-registry-private-configuration\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280897 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/191dfb71-401c-494a-b7bb-cbc6c3273430-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.280970 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d26b56ed-c6e3-4321-8044-d29e5fe081c8-serving-cert\") pod \"service-ca-operator-544d749c8c-kqsd9\" (UID: \"d26b56ed-c6e3-4321-8044-d29e5fe081c8\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281004 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281027 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-cni-multus\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281048 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqqt4\" (UniqueName: \"kubernetes.io/projected/5704adc4-5d8d-439e-ae45-2ab73b06bf14-kube-api-access-vqqt4\") pod \"ingress-canary-767sh\" (UID: \"5704adc4-5d8d-439e-ae45-2ab73b06bf14\") " pod="openshift-ingress-canary/ingress-canary-767sh" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281057 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-image-registry-private-configuration\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281063 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/26ae8a36-06bf-414e-b2bd-325033800b72-dbus\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281107 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x87gt\" (UniqueName: \"kubernetes.io/projected/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-kube-api-access-x87gt\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281140 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpww4\" (UniqueName: \"kubernetes.io/projected/38b751da-304f-4000-bf56-85e1d9aa8152-kube-api-access-cpww4\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281166 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-host\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281192 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-image-registry-private-configuration\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281220 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-bound-sa-token\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281263 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-certificates\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281284 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-modprobe-d\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281302 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90fb287e-4af9-4e0b-919d-00713b37605c-service-ca-bundle\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281329 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-var-lib-kubelet\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281369 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281392 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cf77b4d-32f5-42db-851f-3466ecf583c9-host-slash\") pod \"iptables-alerter-q7mtr\" (UID: \"6cf77b4d-32f5-42db-851f-3466ecf583c9\") " pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281415 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-image-registry-private-configuration\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281419 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fbtn\" (UniqueName: \"kubernetes.io/projected/e10001da-2e9e-4b80-b6e0-0f1fca3df854-kube-api-access-2fbtn\") pod \"kas-connection-checker-685895c7c9-h5ccq\" (UID: \"e10001da-2e9e-4b80-b6e0-0f1fca3df854\") " pod="kube-system/kas-connection-checker-685895c7c9-h5ccq" Nov 04 05:27:11.281479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281464 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-sys-fs\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281509 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72dd8eb9-265d-4761-a507-d91e29606698-ovn-node-metrics-cert\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281545 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qw4k\" (UniqueName: \"kubernetes.io/projected/77f9e498-8338-4f89-ae13-e72e0108a602-kube-api-access-2qw4k\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281642 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/328415b5-7eda-429c-a74a-af29ba790b75-data-volume\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281751 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/328415b5-7eda-429c-a74a-af29ba790b75-data-volume\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281761 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281790 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77f9e498-8338-4f89-ae13-e72e0108a602-cni-binary-copy\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281808 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-kubernetes\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281822 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-run\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281841 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-etc-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281885 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-system-cni-dir\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281911 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281963 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-slash\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.281997 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2flm\" (UniqueName: \"kubernetes.io/projected/523aff5c-ac2b-43be-8e13-792f44b2c14e-kube-api-access-c2flm\") pod \"kas-connection-checker-685895c7c9-5zj4b\" (UID: \"523aff5c-ac2b-43be-8e13-792f44b2c14e\") " pod="kube-system/kas-connection-checker-685895c7c9-5zj4b" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282025 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfptj\" (UniqueName: \"kubernetes.io/projected/191dfb71-401c-494a-b7bb-cbc6c3273430-kube-api-access-gfptj\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282080 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d5fj\" (UniqueName: \"kubernetes.io/projected/6cf77b4d-32f5-42db-851f-3466ecf583c9-kube-api-access-8d5fj\") pod \"iptables-alerter-q7mtr\" (UID: \"6cf77b4d-32f5-42db-851f-3466ecf583c9\") " pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282186 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a35474ea-79b9-4d84-81b1-39db689685d2-nginx-conf\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282206 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-k8s-cni-cncf-io\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282222 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-tuned\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282237 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-device-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282257 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trzdq\" (UniqueName: \"kubernetes.io/projected/72dd8eb9-265d-4761-a507-d91e29606698-kube-api-access-trzdq\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282290 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282343 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz2hq\" (UniqueName: \"kubernetes.io/projected/c87f7dad-1181-4db5-b846-05d8fb0a33d5-kube-api-access-tz2hq\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282361 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-sys\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282362 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-tuned\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.282364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282375 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282409 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282440 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-cni-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282472 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtv2t\" (UniqueName: \"kubernetes.io/projected/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-kube-api-access-dtv2t\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-29hbl\" (UID: \"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282544 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-node-log\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282570 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-cni-bin\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282631 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/26ae8a36-06bf-414e-b2bd-325033800b72-kubelet-config\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282658 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-cnibin\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282701 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-cni-bin\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282725 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-hostroot\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282746 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-bound-sa-token\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282779 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-kubelet\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282813 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/90fb287e-4af9-4e0b-919d-00713b37605c-serving-cert\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282850 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-registration-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282891 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4qpn\" (UniqueName: \"kubernetes.io/projected/328415b5-7eda-429c-a74a-af29ba790b75-kube-api-access-p4qpn\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282921 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7488\" (UniqueName: \"kubernetes.io/projected/90fb287e-4af9-4e0b-919d-00713b37605c-kube-api-access-m7488\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282934 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/90fb287e-4af9-4e0b-919d-00713b37605c-serving-cert\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.282937 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-trusted-ca\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283029 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e7c38cd-3943-484d-b433-cbec7bce35c6-serviceca\") pod \"node-ca-m9v8n\" (UID: \"5e7c38cd-3943-484d-b433-cbec7bce35c6\") " pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283113 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-var-lib-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283186 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/428ba86e-0dd8-4a0a-b899-c7e9d568487c-tmp-dir\") pod \"node-resolver-87f78\" (UID: \"428ba86e-0dd8-4a0a-b899-c7e9d568487c\") " pod="openshift-dns/node-resolver-87f78" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283206 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-config\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283223 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rvs5\" (UniqueName: \"kubernetes.io/projected/b25ddbfa-3e3e-49fd-870f-963d087f22a5-kube-api-access-5rvs5\") pod \"network-check-source-6f78d59f99-xk9lw\" (UID: \"b25ddbfa-3e3e-49fd-870f-963d087f22a5\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xk9lw" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283234 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/428ba86e-0dd8-4a0a-b899-c7e9d568487c-tmp-dir\") pod \"node-resolver-87f78\" (UID: \"428ba86e-0dd8-4a0a-b899-c7e9d568487c\") " pod="openshift-dns/node-resolver-87f78" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283281 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbg2j\" (UniqueName: \"kubernetes.io/projected/886edd3a-6e51-4ccf-92a8-9d8a3d447449-kube-api-access-hbg2j\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283316 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283344 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/328415b5-7eda-429c-a74a-af29ba790b75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283368 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.283391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283393 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-system-cni-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283433 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-netns\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283468 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-etc-kubernetes\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283492 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-ovnkube-script-lib\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283509 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-29hbl\" (UID: \"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283532 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283580 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbx8t\" (UniqueName: \"kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t\") pod \"network-check-target-27hf6\" (UID: \"4a5449c3-4f21-44d2-9d10-9a0956aefdcf\") " pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283617 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-29hbl\" (UID: \"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283620 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-default-certificate\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283651 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283696 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-installation-pull-secrets\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283731 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjngp\" (UniqueName: \"kubernetes.io/projected/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-kube-api-access-xjngp\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283776 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5704adc4-5d8d-439e-ae45-2ab73b06bf14-cert\") pod \"ingress-canary-767sh\" (UID: \"5704adc4-5d8d-439e-ae45-2ab73b06bf14\") " pod="openshift-ingress-canary/ingress-canary-767sh" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283793 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f7d857f-84f9-4bde-9b5a-66164c90377e-tmp\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283808 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e7c38cd-3943-484d-b433-cbec7bce35c6-host\") pod \"node-ca-m9v8n\" (UID: \"5e7c38cd-3943-484d-b433-cbec7bce35c6\") " pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283824 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-systemd-units\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283839 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-multus-certs\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283893 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-default-certificate\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283901 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-installation-pull-secrets\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.284358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.283949 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f7d857f-84f9-4bde-9b5a-66164c90377e-tmp\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.315619 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.315568 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cbfrj\"" Nov 04 05:27:11.315942 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.315879 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 05:27:11.316002 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.315942 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qgv9m\"" Nov 04 05:27:11.316002 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.315946 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-x7j27\"" Nov 04 05:27:11.316002 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.315984 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-2psbf\"" Nov 04 05:27:11.316100 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.316030 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 05:27:11.316660 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.316631 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 05:27:11.316817 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.316756 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 05:27:11.316817 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.316767 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 05:27:11.316817 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.316808 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-bnzsz\"" Nov 04 05:27:11.317056 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.316848 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tb5s5\"" Nov 04 05:27:11.317056 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.316947 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ptjqz\"" Nov 04 05:27:11.317056 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.316978 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vw645\"" Nov 04 05:27:11.317056 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.317047 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fz9sj\"" Nov 04 05:27:11.317290 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.317084 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-8lkk9\"" Nov 04 05:27:11.317290 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.317135 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 05:27:11.317290 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.317135 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6nvfw\"" Nov 04 05:27:11.317290 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.317139 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6qrg6\"" Nov 04 05:27:11.317290 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.317178 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-k2gkz\"" Nov 04 05:27:11.317290 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.317256 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z6mlw\"" Nov 04 05:27:11.317447 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.317350 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 05:27:11.319630 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.319615 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26ae8a36-06bf-414e-b2bd-325033800b72-original-pull-secret\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:27:11.319884 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.319866 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a566754-7023-408d-bb6a-3eb984ffa7e3-agent-certs\") pod \"konnectivity-agent-x27vz\" (UID: \"1a566754-7023-408d-bb6a-3eb984ffa7e3\") " pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:27:11.322081 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.322066 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72dd8eb9-265d-4761-a507-d91e29606698-ovn-node-metrics-cert\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.324307 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.324292 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5704adc4-5d8d-439e-ae45-2ab73b06bf14-cert\") pod \"ingress-canary-767sh\" (UID: \"5704adc4-5d8d-439e-ae45-2ab73b06bf14\") " pod="openshift-ingress-canary/ingress-canary-767sh" Nov 04 05:27:11.338596 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.338574 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6gpf\" (UniqueName: \"kubernetes.io/projected/2e2ba652-6141-4984-8384-51f9720a02ef-kube-api-access-m6gpf\") pod \"kas-connection-checker-685895c7c9-pw5zp\" (UID: \"2e2ba652-6141-4984-8384-51f9720a02ef\") " pod="kube-system/kas-connection-checker-685895c7c9-pw5zp" Nov 04 05:27:11.351076 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.351051 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2flm\" (UniqueName: \"kubernetes.io/projected/523aff5c-ac2b-43be-8e13-792f44b2c14e-kube-api-access-c2flm\") pod \"kas-connection-checker-685895c7c9-5zj4b\" (UID: \"523aff5c-ac2b-43be-8e13-792f44b2c14e\") " pod="kube-system/kas-connection-checker-685895c7c9-5zj4b" Nov 04 05:27:11.351148 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.351088 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-bound-sa-token\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.351244 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.351227 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fbtn\" (UniqueName: \"kubernetes.io/projected/e10001da-2e9e-4b80-b6e0-0f1fca3df854-kube-api-access-2fbtn\") pod \"kas-connection-checker-685895c7c9-h5ccq\" (UID: \"e10001da-2e9e-4b80-b6e0-0f1fca3df854\") " pod="kube-system/kas-connection-checker-685895c7c9-h5ccq" Nov 04 05:27:11.351317 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.351261 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-bound-sa-token\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.367975 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.367954 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Nov 04 05:27:11.367975 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.367956 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Nov 04 05:27:11.368111 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.368076 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.368111 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.368089 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.368231 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.368214 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 05:27:11.368291 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.368232 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 05:27:11.368291 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.368281 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 05:27:11.368394 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.368330 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.368449 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.368390 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 05:27:11.368449 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.368393 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.368620 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.368603 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.368620 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.368615 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.368731 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.368688 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.368965 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.368942 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.368965 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.368961 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.369119 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.368993 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.369119 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.368992 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.369119 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369013 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.369119 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369027 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.369119 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369106 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.369333 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369133 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 05:27:11.369333 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369136 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 05:27:11.369333 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369153 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.369333 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369171 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.369333 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369188 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.369333 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369192 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 05:27:11.369333 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369224 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.369333 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369285 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 05:27:11.369641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369346 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 05:27:11.369641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369362 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.369641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369371 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.369641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369377 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Nov 04 05:27:11.369641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369407 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Nov 04 05:27:11.369641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369415 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Nov 04 05:27:11.369641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369418 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.369641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369428 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 05:27:11.369641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369456 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.369641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369554 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.369641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369569 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.369641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369621 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.370106 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369659 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.370106 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369689 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.370106 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369695 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.370106 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369693 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Nov 04 05:27:11.370106 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369712 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Nov 04 05:27:11.370106 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369710 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.370106 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369735 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.370106 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369821 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.370106 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369850 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.370106 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369938 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 05:27:11.370106 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369948 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 05:27:11.370106 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.369993 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Nov 04 05:27:11.370599 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.370426 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-29hbl\" (UID: \"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:27:11.370599 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.370484 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77f9e498-8338-4f89-ae13-e72e0108a602-multus-daemon-config\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.370599 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.370519 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-ovnkube-config\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.370599 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.370528 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 05:27:11.370599 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.370580 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-env-overrides\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.370823 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.370699 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a566754-7023-408d-bb6a-3eb984ffa7e3-konnectivity-ca\") pod \"konnectivity-agent-x27vz\" (UID: \"1a566754-7023-408d-bb6a-3eb984ffa7e3\") " pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:27:11.370823 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.370806 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws4hd\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-kube-api-access-ws4hd\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.370918 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.370824 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x87gt\" (UniqueName: \"kubernetes.io/projected/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-kube-api-access-x87gt\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.370918 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.370846 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-certificates\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.370918 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.370805 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/886edd3a-6e51-4ccf-92a8-9d8a3d447449-config-volume\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:11.371059 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.370934 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6cf77b4d-32f5-42db-851f-3466ecf583c9-iptables-alerter-script\") pod \"iptables-alerter-q7mtr\" (UID: \"6cf77b4d-32f5-42db-851f-3466ecf583c9\") " pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:27:11.371351 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.371326 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qw4k\" (UniqueName: \"kubernetes.io/projected/77f9e498-8338-4f89-ae13-e72e0108a602-kube-api-access-2qw4k\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.371443 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.371425 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d5fj\" (UniqueName: \"kubernetes.io/projected/6cf77b4d-32f5-42db-851f-3466ecf583c9-kube-api-access-8d5fj\") pod \"iptables-alerter-q7mtr\" (UID: \"6cf77b4d-32f5-42db-851f-3466ecf583c9\") " pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:27:11.371443 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.371435 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4qpn\" (UniqueName: \"kubernetes.io/projected/328415b5-7eda-429c-a74a-af29ba790b75-kube-api-access-p4qpn\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:11.371559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.371424 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7x5w\" (UniqueName: \"kubernetes.io/projected/a10e4998-0b1d-4041-a022-9e36300936fa-kube-api-access-n7x5w\") pod \"volume-data-source-validator-58c9f5f46d-j8lxw\" (UID: \"a10e4998-0b1d-4041-a022-9e36300936fa\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j8lxw" Nov 04 05:27:11.371559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.371484 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-certificates\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.371559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.371525 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p6g8\" (UniqueName: \"kubernetes.io/projected/d998c080-e5b9-4de6-af9c-68a8c531257c-kube-api-access-5p6g8\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:11.371725 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.371559 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfptj\" (UniqueName: \"kubernetes.io/projected/191dfb71-401c-494a-b7bb-cbc6c3273430-kube-api-access-gfptj\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:27:11.371725 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.371588 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7488\" (UniqueName: \"kubernetes.io/projected/90fb287e-4af9-4e0b-919d-00713b37605c-kube-api-access-m7488\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:27:11.371820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.371722 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90fb287e-4af9-4e0b-919d-00713b37605c-service-ca-bundle\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:27:11.371886 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.371864 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.372024 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.371998 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpww4\" (UniqueName: \"kubernetes.io/projected/38b751da-304f-4000-bf56-85e1d9aa8152-kube-api-access-cpww4\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.372077 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.372045 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s7c8\" (UniqueName: \"kubernetes.io/projected/428ba86e-0dd8-4a0a-b899-c7e9d568487c-kube-api-access-9s7c8\") pod \"node-resolver-87f78\" (UID: \"428ba86e-0dd8-4a0a-b899-c7e9d568487c\") " pod="openshift-dns/node-resolver-87f78" Nov 04 05:27:11.372077 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.372065 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77f9e498-8338-4f89-ae13-e72e0108a602-cni-binary-copy\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.372172 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.372081 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwj9c\" (UniqueName: \"kubernetes.io/projected/d26b56ed-c6e3-4321-8044-d29e5fe081c8-kube-api-access-xwj9c\") pod \"service-ca-operator-544d749c8c-kqsd9\" (UID: \"d26b56ed-c6e3-4321-8044-d29e5fe081c8\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:27:11.372305 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.372283 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbg2j\" (UniqueName: \"kubernetes.io/projected/886edd3a-6e51-4ccf-92a8-9d8a3d447449-kube-api-access-hbg2j\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:11.372440 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.372419 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqqt4\" (UniqueName: \"kubernetes.io/projected/5704adc4-5d8d-439e-ae45-2ab73b06bf14-kube-api-access-vqqt4\") pod \"ingress-canary-767sh\" (UID: \"5704adc4-5d8d-439e-ae45-2ab73b06bf14\") " pod="openshift-ingress-canary/ingress-canary-767sh" Nov 04 05:27:11.372543 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.372523 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a35474ea-79b9-4d84-81b1-39db689685d2-nginx-conf\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:27:11.372693 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.372630 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:11.372777 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.372717 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.373534 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.373517 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-config\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:27:11.373607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.373520 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e7c38cd-3943-484d-b433-cbec7bce35c6-serviceca\") pod \"node-ca-m9v8n\" (UID: \"5e7c38cd-3943-484d-b433-cbec7bce35c6\") " pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:27:11.373607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.373568 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.373791 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.373744 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72dd8eb9-265d-4761-a507-d91e29606698-ovnkube-script-lib\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.375051 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.375035 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rvs5\" (UniqueName: \"kubernetes.io/projected/b25ddbfa-3e3e-49fd-870f-963d087f22a5-kube-api-access-5rvs5\") pod \"network-check-source-6f78d59f99-xk9lw\" (UID: \"b25ddbfa-3e3e-49fd-870f-963d087f22a5\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xk9lw" Nov 04 05:27:11.376075 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.376057 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trzdq\" (UniqueName: \"kubernetes.io/projected/72dd8eb9-265d-4761-a507-d91e29606698-kube-api-access-trzdq\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.376182 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.376165 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjngp\" (UniqueName: \"kubernetes.io/projected/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-kube-api-access-xjngp\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:27:11.376255 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.376199 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtv2t\" (UniqueName: \"kubernetes.io/projected/b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04-kube-api-access-dtv2t\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-29hbl\" (UID: \"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" Nov 04 05:27:11.376339 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.376324 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz2hq\" (UniqueName: \"kubernetes.io/projected/c87f7dad-1181-4db5-b846-05d8fb0a33d5-kube-api-access-tz2hq\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:11.376390 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.376349 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbx8t\" (UniqueName: \"kubernetes.io/projected/4a5449c3-4f21-44d2-9d10-9a0956aefdcf-kube-api-access-fbx8t\") pod \"network-check-target-27hf6\" (UID: \"4a5449c3-4f21-44d2-9d10-9a0956aefdcf\") " pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:27:11.379054 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.379034 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qghpk\" (UniqueName: \"kubernetes.io/projected/5e7c38cd-3943-484d-b433-cbec7bce35c6-kube-api-access-qghpk\") pod \"node-ca-m9v8n\" (UID: \"5e7c38cd-3943-484d-b433-cbec7bce35c6\") " pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:27:11.379141 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.379081 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjmwm\" (UniqueName: \"kubernetes.io/projected/8f7d857f-84f9-4bde-9b5a-66164c90377e-kube-api-access-hjmwm\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.379333 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.379315 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8prxh\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-kube-api-access-8prxh\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.379448 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.379420 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-484lx\" (UniqueName: \"kubernetes.io/projected/4a509700-2c8d-4113-9121-3ec287223f70-kube-api-access-484lx\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:27:11.379493 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.379454 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d26b56ed-c6e3-4321-8044-d29e5fe081c8-config\") pod \"service-ca-operator-544d749c8c-kqsd9\" (UID: \"d26b56ed-c6e3-4321-8044-d29e5fe081c8\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" Nov 04 05:27:11.384256 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384237 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-ovn\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.384332 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384271 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/428ba86e-0dd8-4a0a-b899-c7e9d568487c-hosts-file\") pod \"node-resolver-87f78\" (UID: \"428ba86e-0dd8-4a0a-b899-c7e9d568487c\") " pod="openshift-dns/node-resolver-87f78" Nov 04 05:27:11.384332 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384289 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-cni-multus\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.384332 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384303 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/26ae8a36-06bf-414e-b2bd-325033800b72-dbus\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:27:11.384454 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384322 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-host\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.384454 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384345 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-ovn\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.384454 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384357 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/428ba86e-0dd8-4a0a-b899-c7e9d568487c-hosts-file\") pod \"node-resolver-87f78\" (UID: \"428ba86e-0dd8-4a0a-b899-c7e9d568487c\") " pod="openshift-dns/node-resolver-87f78" Nov 04 05:27:11.384454 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384363 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-modprobe-d\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.384454 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384364 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-cni-multus\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.384454 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384386 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/26ae8a36-06bf-414e-b2bd-325033800b72-dbus\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:27:11.384454 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384398 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-host\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.384454 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384416 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-var-lib-kubelet\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.384454 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384390 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-var-lib-kubelet\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.384454 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384441 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:11.384454 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384450 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-modprobe-d\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.384454 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384457 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cf77b4d-32f5-42db-851f-3466ecf583c9-host-slash\") pod \"iptables-alerter-q7mtr\" (UID: \"6cf77b4d-32f5-42db-851f-3466ecf583c9\") " pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384480 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-sys-fs\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384499 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384498 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cf77b4d-32f5-42db-851f-3466ecf583c9-host-slash\") pod \"iptables-alerter-q7mtr\" (UID: \"6cf77b4d-32f5-42db-851f-3466ecf583c9\") " pod="openshift-network-operator/iptables-alerter-q7mtr" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.384517 4859 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384513 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-kubernetes\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384545 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-run\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384549 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-sys-fs\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384586 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-etc-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384610 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-system-cni-dir\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.384591 4859 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384621 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-run\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384640 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-etc-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384637 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-kubernetes\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.384624 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls podName:886edd3a-6e51-4ccf-92a8-9d8a3d447449 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:11.884605039 +0000 UTC m=+1.886240750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls") pod "dns-default-brcbz" (UID: "886edd3a-6e51-4ccf-92a8-9d8a3d447449") : secret "dns-default-metrics-tls" not found Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384645 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-system-cni-dir\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.384718 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls podName:d998c080-e5b9-4de6-af9c-68a8c531257c nodeName:}" failed. No retries permitted until 2026-11-04 05:27:11.884692928 +0000 UTC m=+1.886328641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6nlwl" (UID: "d998c080-e5b9-4de6-af9c-68a8c531257c") : secret "samples-operator-tls" not found Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384734 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384751 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-slash\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384812 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-k8s-cni-cncf-io\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384827 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-device-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384835 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-slash\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384842 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-sys\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384863 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.384813 4859 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384893 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-device-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384907 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-k8s-cni-cncf-io\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384915 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384871 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-sys\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.384917 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert podName:a35474ea-79b9-4d84-81b1-39db689685d2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:11.884900246 +0000 UTC m=+1.886535952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pgdqn" (UID: "a35474ea-79b9-4d84-81b1-39db689685d2") : secret "networking-console-plugin-cert" not found Nov 04 05:27:11.384946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384940 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-cni-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384965 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-node-log\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.384992 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-cni-bin\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385001 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-cni-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385017 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/26ae8a36-06bf-414e-b2bd-325033800b72-kubelet-config\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385021 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-node-log\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385040 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-cnibin\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385064 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-cni-bin\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385080 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-cnibin\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385089 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-hostroot\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385095 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-cni-bin\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385079 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/26ae8a36-06bf-414e-b2bd-325033800b72-kubelet-config\") pod \"global-pull-secret-syncer-nk2gk\" (UID: \"26ae8a36-06bf-414e-b2bd-325033800b72\") " pod="kube-system/global-pull-secret-syncer-nk2gk" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385041 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-cni-bin\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385126 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-hostroot\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385200 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-kubelet\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385114 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-var-lib-kubelet\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385532 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-registration-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385548 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-var-lib-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385564 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385577 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/328415b5-7eda-429c-a74a-af29ba790b75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385592 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-system-cni-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385612 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-netns\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385623 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-var-lib-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385627 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-registration-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385634 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-etc-kubernetes\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385679 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-openvswitch\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385691 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-system-cni-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.385707 4859 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.385728 4859 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.385746 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/328415b5-7eda-429c-a74a-af29ba790b75-kube-rbac-proxy-cm podName:328415b5-7eda-429c-a74a-af29ba790b75 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:11.885735617 +0000 UTC m=+1.887371355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/328415b5-7eda-429c-a74a-af29ba790b75-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xfgjz" (UID: "328415b5-7eda-429c-a74a-af29ba790b75") : configmap "kube-rbac-proxy" not found Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385683 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-etc-kubernetes\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.385763 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls podName:191dfb71-401c-494a-b7bb-cbc6c3273430 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:11.885753513 +0000 UTC m=+1.887389219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-hcsmg" (UID: "191dfb71-401c-494a-b7bb-cbc6c3273430") : secret "cluster-monitoring-operator-tls" not found Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385686 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385787 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-netns\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385799 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385827 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e7c38cd-3943-484d-b433-cbec7bce35c6-host\") pod \"node-ca-m9v8n\" (UID: \"5e7c38cd-3943-484d-b433-cbec7bce35c6\") " pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385853 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-systemd-units\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.385867 4859 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385886 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e7c38cd-3943-484d-b433-cbec7bce35c6-host\") pod \"node-ca-m9v8n\" (UID: \"5e7c38cd-3943-484d-b433-cbec7bce35c6\") " pod="openshift-image-registry/node-ca-m9v8n" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385895 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-systemd-units\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.385902 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs podName:4a509700-2c8d-4113-9121-3ec287223f70 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:11.885893408 +0000 UTC m=+1.887529111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs") pod "network-metrics-daemon-55qcs" (UID: "4a509700-2c8d-4113-9121-3ec287223f70") : secret "metrics-daemon-secret" not found Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385931 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-multus-certs\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385944 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-host-run-multus-certs\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.385943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385954 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-systemd\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385978 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-systemd\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.385991 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-systemd\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386015 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-run-systemd\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386003 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386066 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386092 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386092 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysctl-conf\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386121 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cnibin\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.386135 4859 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386165 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.386173 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls podName:328415b5-7eda-429c-a74a-af29ba790b75 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:11.886163462 +0000 UTC m=+1.887799164 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xfgjz" (UID: "328415b5-7eda-429c-a74a-af29ba790b75") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386136 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386187 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef5a3412-cd67-424f-bb1f-1ec122d5e3eb-cnibin\") pod \"multus-additional-cni-plugins-ddzfn\" (UID: \"ef5a3412-cd67-424f-bb1f-1ec122d5e3eb\") " pod="openshift-multus/multus-additional-cni-plugins-ddzfn" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386205 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386209 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysctl-conf\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386223 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-etc-selinux\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386239 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-run-ovn-kubernetes\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386254 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-cni-netd\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386268 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.386282 4859 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386288 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-etc-selinux\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.386295 4859 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6469786477-m4mgb: secret "image-registry-tls" not found Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386304 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-run-ovn-kubernetes\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386321 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-cni-netd\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386326 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-socket-dir-parent\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386283 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-socket-dir-parent\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.386353 4859 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.386363 4859 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8f7d8c7c6-q48ph: secret "image-registry-tls" not found Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.386354 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls podName:00a06b1e-791a-4652-823c-b73672b7e4b2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:11.886341621 +0000 UTC m=+1.887977364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls") pod "image-registry-6469786477-m4mgb" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2") : secret "image-registry-tls" not found Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.386419 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls podName:aa931f6e-a107-4ca3-bb54-e89a91d0f67a nodeName:}" failed. No retries permitted until 2026-11-04 05:27:11.886404966 +0000 UTC m=+1.888040683 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls") pod "image-registry-8f7d8c7c6-q48ph" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a") : secret "image-registry-tls" not found Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386443 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-lib-modules\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386468 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-log-socket\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386492 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysconfig\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386522 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysctl-d\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386533 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-log-socket\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386557 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-socket-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386577 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysconfig\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386584 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-run-netns\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386590 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-lib-modules\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386613 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-run-netns\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386622 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/38b751da-304f-4000-bf56-85e1d9aa8152-socket-dir\") pod \"aws-ebs-csi-driver-node-xfxxn\" (UID: \"38b751da-304f-4000-bf56-85e1d9aa8152\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xfxxn" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386611 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-kubelet\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386646 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72dd8eb9-265d-4761-a507-d91e29606698-host-kubelet\") pod \"ovnkube-node-7tqkj\" (UID: \"72dd8eb9-265d-4761-a507-d91e29606698\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386677 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/328415b5-7eda-429c-a74a-af29ba790b75-crio-socket\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386689 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f7d857f-84f9-4bde-9b5a-66164c90377e-etc-sysctl-d\") pod \"tuned-wrd6s\" (UID: \"8f7d857f-84f9-4bde-9b5a-66164c90377e\") " pod="openshift-cluster-node-tuning-operator/tuned-wrd6s" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386704 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386724 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-conf-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386740 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.386802 4859 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.386819 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:11.886807067 +0000 UTC m=+1.888442776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : configmap references non-existent config key: service-ca.crt Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386835 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77f9e498-8338-4f89-ae13-e72e0108a602-multus-conf-dir\") pod \"multus-vmgxh\" (UID: \"77f9e498-8338-4f89-ae13-e72e0108a602\") " pod="openshift-multus/multus-vmgxh" Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.386850 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:11.886836894 +0000 UTC m=+1.888472612 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : secret "router-metrics-certs-default" not found Nov 04 05:27:11.387400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.386870 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/328415b5-7eda-429c-a74a-af29ba790b75-crio-socket\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:11.388830 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.388003 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Nov 04 05:27:11.391183 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.391167 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/191dfb71-401c-494a-b7bb-cbc6c3273430-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:27:11.419372 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.419341 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Nov 04 05:27:11.419532 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.419513 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Nov 04 05:27:11.419620 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.419599 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Nov 04 05:27:11.419820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.419803 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-trusted-ca\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.421569 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.421547 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90fb287e-4af9-4e0b-919d-00713b37605c-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-b24ft\" (UID: \"90fb287e-4af9-4e0b-919d-00713b37605c\") " pod="openshift-insights/insights-operator-7c4986d449-b24ft" Nov 04 05:27:11.421569 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.421556 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58bf7d20-81e0-4dbc-85f2-32ded8f14e5f-trusted-ca\") pod \"console-operator-7cd56b8dbc-qnfh8\" (UID: \"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:27:11.423867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.423850 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-trusted-ca\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.518213 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.518188 4859 scope.go:122] "RemoveContainer" containerID="28879b3159ff2859835187b11135533fdeb2de72dd6adcc016e7382ffad5bf8b" Nov 04 05:27:11.892050 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.891919 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:11.892050 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.891951 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:11.892050 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.891970 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:27:11.892050 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.892000 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/328415b5-7eda-429c-a74a-af29ba790b75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:11.892359 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892064 4859 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:11.892359 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.892092 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:27:11.892359 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892123 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls podName:886edd3a-6e51-4ccf-92a8-9d8a3d447449 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:12.892106048 +0000 UTC m=+2.893741758 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls") pod "dns-default-brcbz" (UID: "886edd3a-6e51-4ccf-92a8-9d8a3d447449") : secret "dns-default-metrics-tls" not found Nov 04 05:27:11.892359 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892064 4859 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Nov 04 05:27:11.892359 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892154 4859 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Nov 04 05:27:11.892359 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.892156 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:27:11.892359 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892064 4859 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Nov 04 05:27:11.892359 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892224 4859 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Nov 04 05:27:11.892359 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892200 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert podName:a35474ea-79b9-4d84-81b1-39db689685d2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:12.892181427 +0000 UTC m=+2.893817145 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pgdqn" (UID: "a35474ea-79b9-4d84-81b1-39db689685d2") : secret "networking-console-plugin-cert" not found Nov 04 05:27:11.892359 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.892286 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:11.892359 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.892320 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:11.892359 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.892341 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892369 4859 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.892372 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892419 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls podName:328415b5-7eda-429c-a74a-af29ba790b75 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:12.892404332 +0000 UTC m=+2.894040048 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xfgjz" (UID: "328415b5-7eda-429c-a74a-af29ba790b75") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892426 4859 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892436 4859 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892448 4859 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8f7d8c7c6-q48ph: secret "image-registry-tls" not found Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892455 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls podName:191dfb71-401c-494a-b7bb-cbc6c3273430 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:12.892432484 +0000 UTC m=+2.894068205 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-hcsmg" (UID: "191dfb71-401c-494a-b7bb-cbc6c3273430") : secret "cluster-monitoring-operator-tls" not found Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892468 4859 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892486 4859 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6469786477-m4mgb: secret "image-registry-tls" not found Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892489 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs podName:4a509700-2c8d-4113-9121-3ec287223f70 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:12.892476848 +0000 UTC m=+2.894112555 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs") pod "network-metrics-daemon-55qcs" (UID: "4a509700-2c8d-4113-9121-3ec287223f70") : secret "metrics-daemon-secret" not found Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.892501 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/328415b5-7eda-429c-a74a-af29ba790b75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892516 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls podName:d998c080-e5b9-4de6-af9c-68a8c531257c nodeName:}" failed. No retries permitted until 2026-11-04 05:27:12.892507291 +0000 UTC m=+2.894142994 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6nlwl" (UID: "d998c080-e5b9-4de6-af9c-68a8c531257c") : secret "samples-operator-tls" not found Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892531 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:12.892523506 +0000 UTC m=+2.894159209 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : secret "router-metrics-certs-default" not found Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892544 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls podName:aa931f6e-a107-4ca3-bb54-e89a91d0f67a nodeName:}" failed. No retries permitted until 2026-11-04 05:27:12.892537398 +0000 UTC m=+2.894173101 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls") pod "image-registry-8f7d8c7c6-q48ph" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a") : secret "image-registry-tls" not found Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:11.892556 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892607 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls podName:00a06b1e-791a-4652-823c-b73672b7e4b2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:12.89259049 +0000 UTC m=+2.894226194 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls") pod "image-registry-6469786477-m4mgb" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2") : secret "image-registry-tls" not found Nov 04 05:27:11.892804 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:11.892639 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:12.89262771 +0000 UTC m=+2.894263430 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : configmap references non-existent config key: service-ca.crt Nov 04 05:27:12.166395 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.166375 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/2.log" Nov 04 05:27:12.166906 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.166815 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/1.log" Nov 04 05:27:12.166906 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.166854 4859 generic.go:356] "Generic (PLEG): container finished" podID="58bf7d20-81e0-4dbc-85f2-32ded8f14e5f" containerID="e301a1a147393c152458e0742d3611551efe709b4014b7b7e0574a855db4a10a" exitCode=255 Nov 04 05:27:12.167033 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.166931 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" event={"ID":"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f","Type":"ContainerDied","Data":"e301a1a147393c152458e0742d3611551efe709b4014b7b7e0574a855db4a10a"} Nov 04 05:27:12.167033 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.166989 4859 scope.go:122] "RemoveContainer" containerID="28879b3159ff2859835187b11135533fdeb2de72dd6adcc016e7382ffad5bf8b" Nov 04 05:27:12.167532 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.167510 4859 scope.go:122] "RemoveContainer" containerID="e301a1a147393c152458e0742d3611551efe709b4014b7b7e0574a855db4a10a" Nov 04 05:27:12.167723 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.167700 4859 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-qnfh8_openshift-console-operator(58bf7d20-81e0-4dbc-85f2-32ded8f14e5f)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" podUID="58bf7d20-81e0-4dbc-85f2-32ded8f14e5f" Nov 04 05:27:12.328386 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.328358 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:12.901897 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.901857 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:12.901897 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.901906 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:12.902166 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.901923 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:12.902166 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.901946 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:12.902166 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902002 4859 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Nov 04 05:27:12.902166 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.902032 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:12.902166 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902063 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.902049 +0000 UTC m=+4.903684731 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : secret "router-metrics-certs-default" not found Nov 04 05:27:12.902166 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902007 4859 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:27:12.902166 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902080 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.902074459 +0000 UTC m=+4.903710161 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : configmap references non-existent config key: service-ca.crt Nov 04 05:27:12.902166 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902089 4859 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8f7d8c7c6-q48ph: secret "image-registry-tls" not found Nov 04 05:27:12.902166 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902098 4859 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Nov 04 05:27:12.902166 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.902104 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:27:12.902166 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902126 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls podName:aa931f6e-a107-4ca3-bb54-e89a91d0f67a nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.902111627 +0000 UTC m=+4.903747331 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls") pod "image-registry-8f7d8c7c6-q48ph" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a") : secret "image-registry-tls" not found Nov 04 05:27:12.902166 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902007 4859 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:12.902166 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902147 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls podName:d998c080-e5b9-4de6-af9c-68a8c531257c nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.902139827 +0000 UTC m=+4.903775529 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6nlwl" (UID: "d998c080-e5b9-4de6-af9c-68a8c531257c") : secret "samples-operator-tls" not found Nov 04 05:27:12.902166 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902149 4859 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Nov 04 05:27:12.902166 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902162 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls podName:886edd3a-6e51-4ccf-92a8-9d8a3d447449 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.90215431 +0000 UTC m=+4.903790013 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls") pod "dns-default-brcbz" (UID: "886edd3a-6e51-4ccf-92a8-9d8a3d447449") : secret "dns-default-metrics-tls" not found Nov 04 05:27:12.902714 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902179 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert podName:a35474ea-79b9-4d84-81b1-39db689685d2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.902171156 +0000 UTC m=+4.903806862 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pgdqn" (UID: "a35474ea-79b9-4d84-81b1-39db689685d2") : secret "networking-console-plugin-cert" not found Nov 04 05:27:12.902714 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.902219 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:27:12.902714 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.902248 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:27:12.902714 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.902283 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:12.902714 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:12.902315 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:12.902714 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902321 4859 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Nov 04 05:27:12.902714 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902347 4859 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Nov 04 05:27:12.902714 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902353 4859 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:12.902714 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902367 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls podName:191dfb71-401c-494a-b7bb-cbc6c3273430 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.902354261 +0000 UTC m=+4.903989981 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-hcsmg" (UID: "191dfb71-401c-494a-b7bb-cbc6c3273430") : secret "cluster-monitoring-operator-tls" not found Nov 04 05:27:12.902714 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902385 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs podName:4a509700-2c8d-4113-9121-3ec287223f70 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.902374588 +0000 UTC m=+4.904010297 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs") pod "network-metrics-daemon-55qcs" (UID: "4a509700-2c8d-4113-9121-3ec287223f70") : secret "metrics-daemon-secret" not found Nov 04 05:27:12.902714 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902399 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls podName:328415b5-7eda-429c-a74a-af29ba790b75 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.902391275 +0000 UTC m=+4.904026980 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xfgjz" (UID: "328415b5-7eda-429c-a74a-af29ba790b75") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:12.902714 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902400 4859 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:27:12.902714 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902413 4859 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6469786477-m4mgb: secret "image-registry-tls" not found Nov 04 05:27:12.902714 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:12.902436 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls podName:00a06b1e-791a-4652-823c-b73672b7e4b2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.902429863 +0000 UTC m=+4.904065566 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls") pod "image-registry-6469786477-m4mgb" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2") : secret "image-registry-tls" not found Nov 04 05:27:13.171105 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:13.171083 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/2.log" Nov 04 05:27:14.922037 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:14.922003 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:27:14.922037 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:14.922038 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:14.922063 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922151 4859 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:14.922204 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922211 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls podName:191dfb71-401c-494a-b7bb-cbc6c3273430 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:18.922197472 +0000 UTC m=+8.923833200 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-hcsmg" (UID: "191dfb71-401c-494a-b7bb-cbc6c3273430") : secret "cluster-monitoring-operator-tls" not found Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922154 4859 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:14.922255 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922294 4859 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922300 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls podName:328415b5-7eda-429c-a74a-af29ba790b75 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:18.922284078 +0000 UTC m=+8.923919804 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xfgjz" (UID: "328415b5-7eda-429c-a74a-af29ba790b75") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922306 4859 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6469786477-m4mgb: secret "image-registry-tls" not found Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922154 4859 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:14.922333 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922341 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls podName:00a06b1e-791a-4652-823c-b73672b7e4b2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:18.922328781 +0000 UTC m=+8.923964496 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls") pod "image-registry-6469786477-m4mgb" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2") : secret "image-registry-tls" not found Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922351 4859 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922365 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs podName:4a509700-2c8d-4113-9121-3ec287223f70 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:18.922355215 +0000 UTC m=+8.923990932 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs") pod "network-metrics-daemon-55qcs" (UID: "4a509700-2c8d-4113-9121-3ec287223f70") : secret "metrics-daemon-secret" not found Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:14.922382 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922393 4859 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:14.922414 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922365 4859 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8f7d8c7c6-q48ph: secret "image-registry-tls" not found Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922468 4859 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922422 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:18.922411464 +0000 UTC m=+8.924047167 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : secret "router-metrics-certs-default" not found Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922505 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls podName:aa931f6e-a107-4ca3-bb54-e89a91d0f67a nodeName:}" failed. No retries permitted until 2026-11-04 05:27:18.922487713 +0000 UTC m=+8.924123425 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls") pod "image-registry-8f7d8c7c6-q48ph" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a") : secret "image-registry-tls" not found Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922519 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls podName:886edd3a-6e51-4ccf-92a8-9d8a3d447449 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:18.922513467 +0000 UTC m=+8.924149170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls") pod "dns-default-brcbz" (UID: "886edd3a-6e51-4ccf-92a8-9d8a3d447449") : secret "dns-default-metrics-tls" not found Nov 04 05:27:14.922546 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922536 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle podName:c87f7dad-1181-4db5-b846-05d8fb0a33d5 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:18.922529638 +0000 UTC m=+8.924165341 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle") pod "router-default-58c967f969-4xztc" (UID: "c87f7dad-1181-4db5-b846-05d8fb0a33d5") : configmap references non-existent config key: service-ca.crt Nov 04 05:27:14.923287 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:14.922569 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:14.923287 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:14.922592 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:27:14.923287 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922688 4859 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Nov 04 05:27:14.923287 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922690 4859 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Nov 04 05:27:14.923287 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922718 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert podName:a35474ea-79b9-4d84-81b1-39db689685d2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:18.922711183 +0000 UTC m=+8.924346886 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pgdqn" (UID: "a35474ea-79b9-4d84-81b1-39db689685d2") : secret "networking-console-plugin-cert" not found Nov 04 05:27:14.923287 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:14.922728 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls podName:d998c080-e5b9-4de6-af9c-68a8c531257c nodeName:}" failed. No retries permitted until 2026-11-04 05:27:18.922723338 +0000 UTC m=+8.924359040 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6nlwl" (UID: "d998c080-e5b9-4de6-af9c-68a8c531257c") : secret "samples-operator-tls" not found Nov 04 05:27:14.983216 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:14.983198 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:14.983448 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:14.983421 4859 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 05:27:15.675289 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:15.675260 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x27vz" Nov 04 05:27:15.974471 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:15.974399 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-87f78_428ba86e-0dd8-4a0a-b899-c7e9d568487c/dns-node-resolver/0.log" Nov 04 05:27:16.412374 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:16.412304 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tqkj" Nov 04 05:27:16.779159 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:16.779134 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:27:17.171772 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:17.171748 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m9v8n_5e7c38cd-3943-484d-b433-cbec7bce35c6/node-ca/0.log" Nov 04 05:27:17.573638 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:17.573560 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-767sh_5704adc4-5d8d-439e-ae45-2ab73b06bf14/serve-healthcheck-canary/0.log" Nov 04 05:27:18.163640 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.163609 4859 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:27:18.163994 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.163973 4859 scope.go:122] "RemoveContainer" containerID="e301a1a147393c152458e0742d3611551efe709b4014b7b7e0574a855db4a10a" Nov 04 05:27:18.164173 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:18.164156 4859 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-qnfh8_openshift-console-operator(58bf7d20-81e0-4dbc-85f2-32ded8f14e5f)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" podUID="58bf7d20-81e0-4dbc-85f2-32ded8f14e5f" Nov 04 05:27:18.574388 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.574312 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-29hbl_b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04/kube-storage-version-migrator-operator/0.log" Nov 04 05:27:18.961879 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.961843 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:18.962091 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.961886 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:18.962091 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.961912 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:18.962091 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.961936 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:18.962091 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.961975 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:27:18.962091 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.962016 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:27:18.962091 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.962039 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:27:18.962091 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.962090 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:18.962442 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:18.962100 4859 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Nov 04 05:27:18.962442 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:18.962143 4859 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Nov 04 05:27:18.962442 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:18.962173 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls podName:d998c080-e5b9-4de6-af9c-68a8c531257c nodeName:}" failed. No retries permitted until 2026-11-04 05:27:26.962153025 +0000 UTC m=+16.963788749 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6nlwl" (UID: "d998c080-e5b9-4de6-af9c-68a8c531257c") : secret "samples-operator-tls" not found Nov 04 05:27:18.962442 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:18.962215 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert podName:a35474ea-79b9-4d84-81b1-39db689685d2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:26.96219285 +0000 UTC m=+16.963828553 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pgdqn" (UID: "a35474ea-79b9-4d84-81b1-39db689685d2") : secret "networking-console-plugin-cert" not found Nov 04 05:27:18.962442 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:18.962281 4859 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:18.962442 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.962307 4859 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 05:27:18.962442 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:18.962323 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls podName:328415b5-7eda-429c-a74a-af29ba790b75 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:26.96231011 +0000 UTC m=+16.963945817 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xfgjz" (UID: "328415b5-7eda-429c-a74a-af29ba790b75") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:18.962442 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:18.962356 4859 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:27:18.962442 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:18.962367 4859 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6469786477-m4mgb: secret "image-registry-tls" not found Nov 04 05:27:18.962442 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:18.962394 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls podName:00a06b1e-791a-4652-823c-b73672b7e4b2 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:26.962385692 +0000 UTC m=+16.964021395 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls") pod "image-registry-6469786477-m4mgb" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2") : secret "image-registry-tls" not found Nov 04 05:27:18.962986 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.962542 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:18.962986 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.962608 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:18.962986 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.962717 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c87f7dad-1181-4db5-b846-05d8fb0a33d5-service-ca-bundle\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:18.962986 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:18.962740 4859 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Nov 04 05:27:18.962986 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:18.962756 4859 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8f7d8c7c6-q48ph: secret "image-registry-tls" not found Nov 04 05:27:18.962986 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:18.962807 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls podName:aa931f6e-a107-4ca3-bb54-e89a91d0f67a nodeName:}" failed. No retries permitted until 2026-11-04 05:27:26.962792091 +0000 UTC m=+16.964427810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls") pod "image-registry-8f7d8c7c6-q48ph" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a") : secret "image-registry-tls" not found Nov 04 05:27:18.964613 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.964586 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/191dfb71-401c-494a-b7bb-cbc6c3273430-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-hcsmg\" (UID: \"191dfb71-401c-494a-b7bb-cbc6c3273430\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:27:18.964748 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.964733 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c87f7dad-1181-4db5-b846-05d8fb0a33d5-metrics-certs\") pod \"router-default-58c967f969-4xztc\" (UID: \"c87f7dad-1181-4db5-b846-05d8fb0a33d5\") " pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:18.964852 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.964834 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/886edd3a-6e51-4ccf-92a8-9d8a3d447449-metrics-tls\") pod \"dns-default-brcbz\" (UID: \"886edd3a-6e51-4ccf-92a8-9d8a3d447449\") " pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:18.964922 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:18.964907 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a509700-2c8d-4113-9121-3ec287223f70-metrics-certs\") pod \"network-metrics-daemon-55qcs\" (UID: \"4a509700-2c8d-4113-9121-3ec287223f70\") " pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:27:19.017201 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:19.017176 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" Nov 04 05:27:19.017201 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:19.017199 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:19.021338 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:19.021322 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55qcs" Nov 04 05:27:19.031341 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:19.031314 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:19.398604 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:19.398526 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg"] Nov 04 05:27:19.432134 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:19.432104 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod191dfb71_401c_494a_b7bb_cbc6c3273430.slice/crio-b697ae8665840765e93df2ae19001d324bd97fbafa66ed2f0265bf16fa754262 WatchSource:0}: Error finding container b697ae8665840765e93df2ae19001d324bd97fbafa66ed2f0265bf16fa754262: Status 404 returned error can't find the container with id b697ae8665840765e93df2ae19001d324bd97fbafa66ed2f0265bf16fa754262 Nov 04 05:27:19.434089 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:19.434069 4859 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:27:19.443382 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:19.443360 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a509700_2c8d_4113_9121_3ec287223f70.slice/crio-28dd9cb45389a4b4150f8cf5e6532f05b2fe4b73138bb6fd34bbe6d7d3e4afb8 WatchSource:0}: Error finding container 28dd9cb45389a4b4150f8cf5e6532f05b2fe4b73138bb6fd34bbe6d7d3e4afb8: Status 404 returned error can't find the container with id 28dd9cb45389a4b4150f8cf5e6532f05b2fe4b73138bb6fd34bbe6d7d3e4afb8 Nov 04 05:27:19.443931 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:19.443911 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-55qcs"] Nov 04 05:27:19.447391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:19.447369 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58c967f969-4xztc"] Nov 04 05:27:19.479083 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:19.479058 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc87f7dad_1181_4db5_b846_05d8fb0a33d5.slice/crio-5b9b2625af4e6cb7f896afa3eef3406d2eacffaf2410f46cb6d20770578e2c8c WatchSource:0}: Error finding container 5b9b2625af4e6cb7f896afa3eef3406d2eacffaf2410f46cb6d20770578e2c8c: Status 404 returned error can't find the container with id 5b9b2625af4e6cb7f896afa3eef3406d2eacffaf2410f46cb6d20770578e2c8c Nov 04 05:27:19.494041 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:19.494019 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-brcbz"] Nov 04 05:27:19.494252 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:19.494234 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod886edd3a_6e51_4ccf_92a8_9d8a3d447449.slice/crio-ff7951054f603dcb85797c98901c12a681a4ed8e1af463b6d3ef2689c7a137b9 WatchSource:0}: Error finding container ff7951054f603dcb85797c98901c12a681a4ed8e1af463b6d3ef2689c7a137b9: Status 404 returned error can't find the container with id ff7951054f603dcb85797c98901c12a681a4ed8e1af463b6d3ef2689c7a137b9 Nov 04 05:27:20.198415 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:20.198359 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58c967f969-4xztc" event={"ID":"c87f7dad-1181-4db5-b846-05d8fb0a33d5","Type":"ContainerStarted","Data":"7cda1cd7290d77e4e94a03c68ed948c327bd585d114c0429dc7e6e49430e2cc1"} Nov 04 05:27:20.198910 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:20.198432 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58c967f969-4xztc" event={"ID":"c87f7dad-1181-4db5-b846-05d8fb0a33d5","Type":"ContainerStarted","Data":"5b9b2625af4e6cb7f896afa3eef3406d2eacffaf2410f46cb6d20770578e2c8c"} Nov 04 05:27:20.199726 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:20.199693 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-brcbz" event={"ID":"886edd3a-6e51-4ccf-92a8-9d8a3d447449","Type":"ContainerStarted","Data":"ff7951054f603dcb85797c98901c12a681a4ed8e1af463b6d3ef2689c7a137b9"} Nov 04 05:27:20.200858 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:20.200835 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-55qcs" event={"ID":"4a509700-2c8d-4113-9121-3ec287223f70","Type":"ContainerStarted","Data":"28dd9cb45389a4b4150f8cf5e6532f05b2fe4b73138bb6fd34bbe6d7d3e4afb8"} Nov 04 05:27:20.202050 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:20.202024 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" event={"ID":"191dfb71-401c-494a-b7bb-cbc6c3273430","Type":"ContainerStarted","Data":"b697ae8665840765e93df2ae19001d324bd97fbafa66ed2f0265bf16fa754262"} Nov 04 05:27:20.691182 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:20.691145 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:27:20.691586 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:20.691568 4859 scope.go:122] "RemoveContainer" containerID="e301a1a147393c152458e0742d3611551efe709b4014b7b7e0574a855db4a10a" Nov 04 05:27:20.691829 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:20.691803 4859 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-qnfh8_openshift-console-operator(58bf7d20-81e0-4dbc-85f2-32ded8f14e5f)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" podUID="58bf7d20-81e0-4dbc-85f2-32ded8f14e5f" Nov 04 05:27:20.877346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:20.877317 4859 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-143.us-west-2.compute.internal" event="NodeReady" Nov 04 05:27:21.018011 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:21.017928 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:21.021323 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:21.021294 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:22.109608 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:22.109583 4859 scope.go:122] "RemoveContainer" containerID="e301a1a147393c152458e0742d3611551efe709b4014b7b7e0574a855db4a10a" Nov 04 05:27:22.215642 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:22.214416 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-55qcs" event={"ID":"4a509700-2c8d-4113-9121-3ec287223f70","Type":"ContainerStarted","Data":"b7f889e6a921487ed224f1e3901500970e57bcc5dde9602bb82e0b8841dedda8"} Nov 04 05:27:22.215642 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:22.214490 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-55qcs" event={"ID":"4a509700-2c8d-4113-9121-3ec287223f70","Type":"ContainerStarted","Data":"f138925d5e416eefe4171dbab8d0c9642d89ff14eaf0b4a484575393aa505fb2"} Nov 04 05:27:22.217237 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:22.217210 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-hcsmg" event={"ID":"191dfb71-401c-494a-b7bb-cbc6c3273430","Type":"ContainerStarted","Data":"9ca9ac6adbced788e0f18aac9a0e0b535bf816ef4d5134fc1578c5d50b2fa4e0"} Nov 04 05:27:22.220564 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:22.219270 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/2.log" Nov 04 05:27:22.220564 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:22.219406 4859 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 05:27:22.220564 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:22.219828 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" event={"ID":"58bf7d20-81e0-4dbc-85f2-32ded8f14e5f","Type":"ContainerStarted","Data":"48bb8a263c20af26c8f95eba1b6aebbed48dd318aa2de1e21c183eb8639a6ac2"} Nov 04 05:27:22.220564 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:22.220497 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:27:22.221606 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:22.221492 4859 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-qnfh8 container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.13:8443/readyz\": dial tcp 10.132.0.13:8443: connect: connection refused" start-of-body= Nov 04 05:27:22.221606 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:22.221537 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" podUID="58bf7d20-81e0-4dbc-85f2-32ded8f14e5f" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.13:8443/readyz\": dial tcp 10.132.0.13:8443: connect: connection refused" Nov 04 05:27:23.224746 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:23.224710 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-brcbz" event={"ID":"886edd3a-6e51-4ccf-92a8-9d8a3d447449","Type":"ContainerStarted","Data":"fd0905b9c4f0ebb3b6c4e77c29ce600ff7ca2af411d8505fe2572ee6503a0c36"} Nov 04 05:27:23.225228 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:23.224769 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-brcbz" event={"ID":"886edd3a-6e51-4ccf-92a8-9d8a3d447449","Type":"ContainerStarted","Data":"17fc431b77c67a5cf74f7e55ca586cbd3a4b6885301312f2ad4890ee39381d62"} Nov 04 05:27:23.225297 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:23.225226 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:23.229028 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:23.229010 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-qnfh8" Nov 04 05:27:27.037773 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.037737 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:27.037773 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.037776 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:27.038318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.037797 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:27.038318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.037856 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:27.038318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.037885 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:27:27.040187 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.040159 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/328415b5-7eda-429c-a74a-af29ba790b75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfgjz\" (UID: \"328415b5-7eda-429c-a74a-af29ba790b75\") " pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:27.040905 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.040880 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\") pod \"image-registry-8f7d8c7c6-q48ph\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:27.041001 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.040886 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d998c080-e5b9-4de6-af9c-68a8c531257c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6nlwl\" (UID: \"d998c080-e5b9-4de6-af9c-68a8c531257c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:27.041039 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.041017 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\") pod \"image-registry-6469786477-m4mgb\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:27.041320 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.041301 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a35474ea-79b9-4d84-81b1-39db689685d2-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pgdqn\" (UID: \"a35474ea-79b9-4d84-81b1-39db689685d2\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:27:27.116318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.116293 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" Nov 04 05:27:27.119443 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.119420 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" Nov 04 05:27:27.119617 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.119492 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:27.126498 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.126480 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:27.130064 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.130039 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xfgjz" Nov 04 05:27:27.420595 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.420559 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:27.420754 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.420717 4859 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 05:27:27.422144 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.422125 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-58c967f969-4xztc" Nov 04 05:27:27.525436 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.525407 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl"] Nov 04 05:27:27.550433 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.550413 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-pgdqn"] Nov 04 05:27:27.556838 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:27.556814 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda35474ea_79b9_4d84_81b1_39db689685d2.slice/crio-d584573db9ef7767064f5ec25bdc94525f577b40c4bac0ec5b4f44c11c0ebf48 WatchSource:0}: Error finding container d584573db9ef7767064f5ec25bdc94525f577b40c4bac0ec5b4f44c11c0ebf48: Status 404 returned error can't find the container with id d584573db9ef7767064f5ec25bdc94525f577b40c4bac0ec5b4f44c11c0ebf48 Nov 04 05:27:27.575280 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.575258 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6469786477-m4mgb"] Nov 04 05:27:27.585541 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:27.585520 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00a06b1e_791a_4652_823c_b73672b7e4b2.slice/crio-8cc1eae264a5e41c11ef9c47accaa49d94e3710d51b6971bba23e8d35df3346f WatchSource:0}: Error finding container 8cc1eae264a5e41c11ef9c47accaa49d94e3710d51b6971bba23e8d35df3346f: Status 404 returned error can't find the container with id 8cc1eae264a5e41c11ef9c47accaa49d94e3710d51b6971bba23e8d35df3346f Nov 04 05:27:27.599700 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:27.599652 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod328415b5_7eda_429c_a74a_af29ba790b75.slice/crio-653de95f488a40d53596cc829bd7f8d054afd259c627aec5f622f22605681d7b WatchSource:0}: Error finding container 653de95f488a40d53596cc829bd7f8d054afd259c627aec5f622f22605681d7b: Status 404 returned error can't find the container with id 653de95f488a40d53596cc829bd7f8d054afd259c627aec5f622f22605681d7b Nov 04 05:27:27.602504 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:27.602477 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa931f6e_a107_4ca3_bb54_e89a91d0f67a.slice/crio-2e8dc128a4f139473a239d7d74c1b36a5221e5d4fcb3fdb20bcd12fb6cfbbcff WatchSource:0}: Error finding container 2e8dc128a4f139473a239d7d74c1b36a5221e5d4fcb3fdb20bcd12fb6cfbbcff: Status 404 returned error can't find the container with id 2e8dc128a4f139473a239d7d74c1b36a5221e5d4fcb3fdb20bcd12fb6cfbbcff Nov 04 05:27:27.607746 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.607722 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xfgjz"] Nov 04 05:27:27.625555 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:27.625532 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8f7d8c7c6-q48ph"] Nov 04 05:27:28.248141 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:28.248106 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" event={"ID":"a35474ea-79b9-4d84-81b1-39db689685d2","Type":"ContainerStarted","Data":"d584573db9ef7767064f5ec25bdc94525f577b40c4bac0ec5b4f44c11c0ebf48"} Nov 04 05:27:28.249744 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:28.249721 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" event={"ID":"aa931f6e-a107-4ca3-bb54-e89a91d0f67a","Type":"ContainerStarted","Data":"23c25ac730a8f54446d8c338499397e5dbcbb6799a9a89c891dcdfea98a9f551"} Nov 04 05:27:28.249861 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:28.249772 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" event={"ID":"aa931f6e-a107-4ca3-bb54-e89a91d0f67a","Type":"ContainerStarted","Data":"2e8dc128a4f139473a239d7d74c1b36a5221e5d4fcb3fdb20bcd12fb6cfbbcff"} Nov 04 05:27:28.249861 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:28.249829 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:28.251438 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:28.251415 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6469786477-m4mgb" event={"ID":"00a06b1e-791a-4652-823c-b73672b7e4b2","Type":"ContainerStarted","Data":"d5238739879c8658be11e88a7d583caf22338cf128ffb8d7411e45d693390626"} Nov 04 05:27:28.251534 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:28.251456 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:28.251534 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:28.251475 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6469786477-m4mgb" event={"ID":"00a06b1e-791a-4652-823c-b73672b7e4b2","Type":"ContainerStarted","Data":"8cc1eae264a5e41c11ef9c47accaa49d94e3710d51b6971bba23e8d35df3346f"} Nov 04 05:27:28.252849 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:28.252830 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xfgjz" event={"ID":"328415b5-7eda-429c-a74a-af29ba790b75","Type":"ContainerStarted","Data":"aff7e92f12620e500410d85fba02aa856f65105f25d8d94651c5106410c543db"} Nov 04 05:27:28.252925 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:28.252868 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xfgjz" event={"ID":"328415b5-7eda-429c-a74a-af29ba790b75","Type":"ContainerStarted","Data":"653de95f488a40d53596cc829bd7f8d054afd259c627aec5f622f22605681d7b"} Nov 04 05:27:28.253890 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:28.253869 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" event={"ID":"d998c080-e5b9-4de6-af9c-68a8c531257c","Type":"ContainerStarted","Data":"4e59e273405dc80560906a517d12abd4ba4fd4d4634455989e44078d21250316"} Nov 04 05:27:29.259012 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:29.258931 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xfgjz" event={"ID":"328415b5-7eda-429c-a74a-af29ba790b75","Type":"ContainerStarted","Data":"454cd905d26fcf2b54aced34eff9a258a72e4adf0e0391375c4e11e38fa9ff57"} Nov 04 05:27:30.263904 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:30.263867 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-pgdqn" event={"ID":"a35474ea-79b9-4d84-81b1-39db689685d2","Type":"ContainerStarted","Data":"67ffb0166b132060075ea8cab4f75a42804c62efc8b503fa02ed6840d8443fee"} Nov 04 05:27:31.271701 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:31.271611 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xfgjz" event={"ID":"328415b5-7eda-429c-a74a-af29ba790b75","Type":"ContainerStarted","Data":"b56d3300f0cc5905bf2f2d9d8bd36b13c672ea329d37829e909abd247e597a24"} Nov 04 05:27:32.038173 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:32.038138 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6469786477-m4mgb"] Nov 04 05:27:34.232788 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:34.232762 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-brcbz" Nov 04 05:27:34.283228 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:34.283193 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" event={"ID":"d998c080-e5b9-4de6-af9c-68a8c531257c","Type":"ContainerStarted","Data":"decf37becbe1457c9c46fb2525afdfcee310c5b9060e22a271885766b5d70174"} Nov 04 05:27:34.283369 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:34.283245 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6nlwl" event={"ID":"d998c080-e5b9-4de6-af9c-68a8c531257c","Type":"ContainerStarted","Data":"08664229886fbbff501819222d5e2565e906f1def181876e7e8dfd3d52e1de87"} Nov 04 05:27:42.044066 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:42.044038 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:43.119005 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.118958 4859 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x27vz_1a566754-7023-408d-bb6a-3eb984ffa7e3/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:43.433230 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.433188 4859 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x27vz_1a566754-7023-408d-bb6a-3eb984ffa7e3/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:43.634392 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.634354 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bbm2w"] Nov 04 05:27:43.667016 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.666987 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.709537 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.709470 4859 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x27vz_1a566754-7023-408d-bb6a-3eb984ffa7e3/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:43.724955 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.724926 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Nov 04 05:27:43.725072 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.724958 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Nov 04 05:27:43.769992 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.769966 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 04 05:27:43.770121 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.770010 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-szc85\"" Nov 04 05:27:43.770121 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.770070 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Nov 04 05:27:43.773403 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.773383 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-node-exporter-tls\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.773502 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.773418 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-root\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.773502 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.773447 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-sys\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.773614 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.773559 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-node-exporter-textfile\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.773614 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.773601 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-node-exporter-accelerators-collector-config\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.773732 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.773634 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-metrics-client-ca\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.773768 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.773741 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-node-exporter-wtmp\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.773810 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.773777 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqzbm\" (UniqueName: \"kubernetes.io/projected/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-kube-api-access-sqzbm\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.773857 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.773838 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.874414 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.874383 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-node-exporter-tls\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.874414 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.874415 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-root\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.874655 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.874436 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-sys\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.874655 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.874459 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-node-exporter-textfile\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.874655 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.874477 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-node-exporter-accelerators-collector-config\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.874655 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.874521 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-metrics-client-ca\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.874655 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.874554 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-sys\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.874655 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.874520 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-root\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.874655 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.874587 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-node-exporter-wtmp\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.874655 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.874625 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqzbm\" (UniqueName: \"kubernetes.io/projected/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-kube-api-access-sqzbm\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.875050 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.874687 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.875050 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.874778 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-node-exporter-wtmp\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.875050 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.874930 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-node-exporter-textfile\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.875185 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.875103 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-node-exporter-accelerators-collector-config\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.875224 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.875177 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-metrics-client-ca\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.877235 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.877201 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.877235 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.877214 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-node-exporter-tls\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:43.935615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:43.935587 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqzbm\" (UniqueName: \"kubernetes.io/projected/ffdcf7c5-6d29-4e42-b8ac-6495001fea2d-kube-api-access-sqzbm\") pod \"node-exporter-bbm2w\" (UID: \"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d\") " pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:44.031907 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:44.031840 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bbm2w" Nov 04 05:27:44.063287 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:44.063258 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffdcf7c5_6d29_4e42_b8ac_6495001fea2d.slice/crio-584d7ef11a7b9ae18261cebc4f2372764aa0f60abf592d9a9c85960c4f7c405c WatchSource:0}: Error finding container 584d7ef11a7b9ae18261cebc4f2372764aa0f60abf592d9a9c85960c4f7c405c: Status 404 returned error can't find the container with id 584d7ef11a7b9ae18261cebc4f2372764aa0f60abf592d9a9c85960c4f7c405c Nov 04 05:27:44.322517 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:44.322441 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bbm2w" event={"ID":"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d","Type":"ContainerStarted","Data":"584d7ef11a7b9ae18261cebc4f2372764aa0f60abf592d9a9c85960c4f7c405c"} Nov 04 05:27:46.331912 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:46.331876 4859 generic.go:356] "Generic (PLEG): container finished" podID="ffdcf7c5-6d29-4e42-b8ac-6495001fea2d" containerID="b684b69bb35c360c4488620d38ca77f5efcebcf4673414e9e600b8397fbc253b" exitCode=0 Nov 04 05:27:46.332314 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:46.331939 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bbm2w" event={"ID":"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d","Type":"ContainerDied","Data":"b684b69bb35c360c4488620d38ca77f5efcebcf4673414e9e600b8397fbc253b"} Nov 04 05:27:46.782530 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:46.782496 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-27hf6" Nov 04 05:27:47.338003 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:47.337971 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bbm2w" event={"ID":"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d","Type":"ContainerStarted","Data":"c203f4aa4d890ad02ccc2ac827f7de24fbe8269858681e33ffc34b07eb44b996"} Nov 04 05:27:47.338436 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:47.338030 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bbm2w" event={"ID":"ffdcf7c5-6d29-4e42-b8ac-6495001fea2d","Type":"ContainerStarted","Data":"7d3c5ef1f15a13eff38af82da5c689ce706c8cbad1aa5aceac2e813cd5f0208f"} Nov 04 05:27:47.461317 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:47.461266 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bbm2w" podStartSLOduration=2.32166923 podStartE2EDuration="4.461251726s" totalImagesPullingTime="2.070120475s" totalInitContainerRuntime="69.462021ms" isStatefulPod=true podCreationTimestamp="2026-11-04 05:27:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:47.460176957 +0000 UTC m=+37.461812681" watchObservedRunningTime="2026-11-04 05:27:47.461251726 +0000 UTC m=+37.462887445" Nov 04 05:27:49.263758 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.263728 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:27:49.367408 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.367366 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-cf64454f5-lp7q5"] Nov 04 05:27:49.369768 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.369751 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.424266 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.424239 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-telemeter-client-tls\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.424381 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.424277 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.424381 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.424361 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-serving-certs-ca-bundle\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.424471 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.424400 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-federate-client-tls\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.424471 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.424426 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.424547 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.424489 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-secret-telemeter-client\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.424547 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.424517 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-metrics-client-ca\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.424547 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.424538 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhj9k\" (UniqueName: \"kubernetes.io/projected/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-kube-api-access-jhj9k\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.428242 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.428222 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Nov 04 05:27:49.428553 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.428535 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-k9xhw\"" Nov 04 05:27:49.428600 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.428577 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Nov 04 05:27:49.433740 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.433715 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-cf64454f5-lp7q5"] Nov 04 05:27:49.478633 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.478604 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Nov 04 05:27:49.525111 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.525042 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-secret-telemeter-client\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.525111 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.525092 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-metrics-client-ca\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.525326 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.525117 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhj9k\" (UniqueName: \"kubernetes.io/projected/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-kube-api-access-jhj9k\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.525326 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.525156 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-telemeter-client-tls\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.525326 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.525193 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.525326 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.525251 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-serving-certs-ca-bundle\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.525326 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.525276 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-federate-client-tls\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.525326 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.525298 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.525938 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.525911 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-metrics-client-ca\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.526021 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.525914 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-serving-certs-ca-bundle\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.527457 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.527436 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-telemeter-client-tls\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.527596 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.527578 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-secret-telemeter-client\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.538903 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.533386 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Nov 04 05:27:49.538903 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.533418 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Nov 04 05:27:49.538903 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.536052 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\"" Nov 04 05:27:49.538903 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.538289 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.539402 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.539382 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-federate-client-tls\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.539709 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.539690 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.586310 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.586278 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhj9k\" (UniqueName: \"kubernetes.io/projected/7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544-kube-api-access-jhj9k\") pod \"telemeter-client-cf64454f5-lp7q5\" (UID: \"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544\") " pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:49.682959 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:49.682922 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" Nov 04 05:27:50.106171 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:27:50.106132 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a1d3ac9_d6ba_4027_aee2_77fc2ff9e544.slice/crio-5a00385c386c5737c4d512ec051e28b2d662282a3b2943f4cebf84b99492e3cb WatchSource:0}: Error finding container 5a00385c386c5737c4d512ec051e28b2d662282a3b2943f4cebf84b99492e3cb: Status 404 returned error can't find the container with id 5a00385c386c5737c4d512ec051e28b2d662282a3b2943f4cebf84b99492e3cb Nov 04 05:27:50.106562 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:50.106405 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-cf64454f5-lp7q5"] Nov 04 05:27:50.349913 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:50.349878 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" event={"ID":"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544","Type":"ContainerStarted","Data":"5a00385c386c5737c4d512ec051e28b2d662282a3b2943f4cebf84b99492e3cb"} Nov 04 05:27:52.178013 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:52.177974 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8f7d8c7c6-q48ph"] Nov 04 05:27:54.367071 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:54.367033 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" event={"ID":"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544","Type":"ContainerStarted","Data":"3142f687eaab726966dded717236143123747db586a8a417c9cbaa11942e0226"} Nov 04 05:27:56.377891 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:56.377799 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" event={"ID":"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544","Type":"ContainerStarted","Data":"d2073d38d57f8786d41ae8037157ef2d0243cc3fa5c133fc38f9accba87694c5"} Nov 04 05:27:56.377891 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:56.377866 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" event={"ID":"7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544","Type":"ContainerStarted","Data":"10261fe24c0022d45f68a81b723d5f16a1a229790bd712e54fb17d0233c17b5a"} Nov 04 05:27:56.599574 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:56.599516 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-cf64454f5-lp7q5" podStartSLOduration=1.7603408090000001 podStartE2EDuration="7.599502864s" totalImagesPullingTime="5.839162055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:56.549204609 +0000 UTC m=+46.550840334" watchObservedRunningTime="2026-11-04 05:27:56.599502864 +0000 UTC m=+46.601138586" Nov 04 05:27:57.058379 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.058341 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6469786477-m4mgb" podUID="00a06b1e-791a-4652-823c-b73672b7e4b2" containerName="registry" containerID="cri-o://d5238739879c8658be11e88a7d583caf22338cf128ffb8d7411e45d693390626" gracePeriod=30 Nov 04 05:27:57.318887 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.318835 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:57.377891 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.377855 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\") pod \"00a06b1e-791a-4652-823c-b73672b7e4b2\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " Nov 04 05:27:57.378318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.377967 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-bound-sa-token\") pod \"00a06b1e-791a-4652-823c-b73672b7e4b2\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " Nov 04 05:27:57.378318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.378010 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00a06b1e-791a-4652-823c-b73672b7e4b2-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/00a06b1e-791a-4652-823c-b73672b7e4b2-ca-trust-extracted\") pod \"00a06b1e-791a-4652-823c-b73672b7e4b2\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " Nov 04 05:27:57.378318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.378048 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-trusted-ca\") pod \"00a06b1e-791a-4652-823c-b73672b7e4b2\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " Nov 04 05:27:57.378318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.378091 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-certificates\") pod \"00a06b1e-791a-4652-823c-b73672b7e4b2\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " Nov 04 05:27:57.378318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.378133 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-kube-api-access-ws4hd\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-kube-api-access-ws4hd\") pod \"00a06b1e-791a-4652-823c-b73672b7e4b2\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " Nov 04 05:27:57.378906 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.378840 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-trusted-ca" pod "00a06b1e-791a-4652-823c-b73672b7e4b2" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:27:57.379254 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.379036 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-certificates" pod "00a06b1e-791a-4652-823c-b73672b7e4b2" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:27:57.385337 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.381351 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls" pod "00a06b1e-791a-4652-823c-b73672b7e4b2" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:27:57.385800 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.385769 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-bound-sa-token" pod "00a06b1e-791a-4652-823c-b73672b7e4b2" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:27:57.386753 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.386726 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-kube-api-access-ws4hd" pod "00a06b1e-791a-4652-823c-b73672b7e4b2" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2"). InnerVolumeSpecName "kube-api-access-ws4hd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:27:57.387650 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.387504 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00a06b1e-791a-4652-823c-b73672b7e4b2-ca-trust-extracted" pod "00a06b1e-791a-4652-823c-b73672b7e4b2" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:27:57.387650 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.387578 4859 generic.go:356] "Generic (PLEG): container finished" podID="00a06b1e-791a-4652-823c-b73672b7e4b2" containerID="d5238739879c8658be11e88a7d583caf22338cf128ffb8d7411e45d693390626" exitCode=0 Nov 04 05:27:57.387650 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.387638 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6469786477-m4mgb" Nov 04 05:27:57.387800 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.387652 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6469786477-m4mgb" event={"ID":"00a06b1e-791a-4652-823c-b73672b7e4b2","Type":"ContainerDied","Data":"d5238739879c8658be11e88a7d583caf22338cf128ffb8d7411e45d693390626"} Nov 04 05:27:57.387800 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.387737 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6469786477-m4mgb" event={"ID":"00a06b1e-791a-4652-823c-b73672b7e4b2","Type":"ContainerDied","Data":"8cc1eae264a5e41c11ef9c47accaa49d94e3710d51b6971bba23e8d35df3346f"} Nov 04 05:27:57.387800 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.387759 4859 scope.go:122] "RemoveContainer" containerID="d5238739879c8658be11e88a7d583caf22338cf128ffb8d7411e45d693390626" Nov 04 05:27:57.398150 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.398131 4859 scope.go:122] "RemoveContainer" containerID="d5238739879c8658be11e88a7d583caf22338cf128ffb8d7411e45d693390626" Nov 04 05:27:57.398427 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:57.398407 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5238739879c8658be11e88a7d583caf22338cf128ffb8d7411e45d693390626\": container with ID starting with d5238739879c8658be11e88a7d583caf22338cf128ffb8d7411e45d693390626 not found: ID does not exist" containerID="d5238739879c8658be11e88a7d583caf22338cf128ffb8d7411e45d693390626" Nov 04 05:27:57.398486 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.398435 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5238739879c8658be11e88a7d583caf22338cf128ffb8d7411e45d693390626"} err="failed to get container status \"d5238739879c8658be11e88a7d583caf22338cf128ffb8d7411e45d693390626\": rpc error: code = NotFound desc = could not find container \"d5238739879c8658be11e88a7d583caf22338cf128ffb8d7411e45d693390626\": container with ID starting with d5238739879c8658be11e88a7d583caf22338cf128ffb8d7411e45d693390626 not found: ID does not exist" Nov 04 05:27:57.479081 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.479047 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-installation-pull-secrets\") pod \"00a06b1e-791a-4652-823c-b73672b7e4b2\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " Nov 04 05:27:57.479413 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.479391 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-image-registry-private-configuration\") pod \"00a06b1e-791a-4652-823c-b73672b7e4b2\" (UID: \"00a06b1e-791a-4652-823c-b73672b7e4b2\") " Nov 04 05:27:57.480115 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.480074 4859 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/00a06b1e-791a-4652-823c-b73672b7e4b2-ca-trust-extracted\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:27:57.480256 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.480120 4859 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-trusted-ca\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:27:57.480256 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.480139 4859 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-certificates\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:27:57.480256 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.480156 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ws4hd\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-kube-api-access-ws4hd\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:27:57.480256 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.480171 4859 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-registry-tls\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:27:57.480256 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.480200 4859 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00a06b1e-791a-4652-823c-b73672b7e4b2-bound-sa-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:27:57.481332 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.481308 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-installation-pull-secrets" pod "00a06b1e-791a-4652-823c-b73672b7e4b2" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:27:57.481781 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.481759 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-image-registry-private-configuration" pod "00a06b1e-791a-4652-823c-b73672b7e4b2" (UID: "00a06b1e-791a-4652-823c-b73672b7e4b2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:27:57.581018 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.580935 4859 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-installation-pull-secrets\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:27:57.581018 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.580968 4859 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/00a06b1e-791a-4652-823c-b73672b7e4b2-image-registry-private-configuration\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:27:57.807953 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.807918 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6469786477-m4mgb"] Nov 04 05:27:57.811572 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:57.811547 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6469786477-m4mgb"] Nov 04 05:27:58.114042 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:27:58.114009 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00a06b1e-791a-4652-823c-b73672b7e4b2" path="/var/lib/kubelet/pods/00a06b1e-791a-4652-823c-b73672b7e4b2/volumes" Nov 04 05:27:58.340487 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:27:58.340406 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:28:17.198382 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.198345 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" podUID="aa931f6e-a107-4ca3-bb54-e89a91d0f67a" containerName="registry" containerID="cri-o://23c25ac730a8f54446d8c338499397e5dbcbb6799a9a89c891dcdfea98a9f551" gracePeriod=30 Nov 04 05:28:17.452203 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.452177 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:28:17.465161 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.465141 4859 generic.go:356] "Generic (PLEG): container finished" podID="b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04" containerID="593a8e6fdd494057081982351a782a2582bf25b804a1f2170082eea0914cc7e7" exitCode=0 Nov 04 05:28:17.465273 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.465221 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" event={"ID":"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04","Type":"ContainerDied","Data":"593a8e6fdd494057081982351a782a2582bf25b804a1f2170082eea0914cc7e7"} Nov 04 05:28:17.465579 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.465563 4859 scope.go:122] "RemoveContainer" containerID="593a8e6fdd494057081982351a782a2582bf25b804a1f2170082eea0914cc7e7" Nov 04 05:28:17.466500 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.466476 4859 generic.go:356] "Generic (PLEG): container finished" podID="aa931f6e-a107-4ca3-bb54-e89a91d0f67a" containerID="23c25ac730a8f54446d8c338499397e5dbcbb6799a9a89c891dcdfea98a9f551" exitCode=0 Nov 04 05:28:17.466591 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.466515 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" event={"ID":"aa931f6e-a107-4ca3-bb54-e89a91d0f67a","Type":"ContainerDied","Data":"23c25ac730a8f54446d8c338499397e5dbcbb6799a9a89c891dcdfea98a9f551"} Nov 04 05:28:17.466591 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.466518 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" Nov 04 05:28:17.466591 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.466547 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8f7d8c7c6-q48ph" event={"ID":"aa931f6e-a107-4ca3-bb54-e89a91d0f67a","Type":"ContainerDied","Data":"2e8dc128a4f139473a239d7d74c1b36a5221e5d4fcb3fdb20bcd12fb6cfbbcff"} Nov 04 05:28:17.466591 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.466581 4859 scope.go:122] "RemoveContainer" containerID="23c25ac730a8f54446d8c338499397e5dbcbb6799a9a89c891dcdfea98a9f551" Nov 04 05:28:17.477349 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.477330 4859 scope.go:122] "RemoveContainer" containerID="23c25ac730a8f54446d8c338499397e5dbcbb6799a9a89c891dcdfea98a9f551" Nov 04 05:28:17.477610 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:28:17.477593 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23c25ac730a8f54446d8c338499397e5dbcbb6799a9a89c891dcdfea98a9f551\": container with ID starting with 23c25ac730a8f54446d8c338499397e5dbcbb6799a9a89c891dcdfea98a9f551 not found: ID does not exist" containerID="23c25ac730a8f54446d8c338499397e5dbcbb6799a9a89c891dcdfea98a9f551" Nov 04 05:28:17.477676 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.477618 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23c25ac730a8f54446d8c338499397e5dbcbb6799a9a89c891dcdfea98a9f551"} err="failed to get container status \"23c25ac730a8f54446d8c338499397e5dbcbb6799a9a89c891dcdfea98a9f551\": rpc error: code = NotFound desc = could not find container \"23c25ac730a8f54446d8c338499397e5dbcbb6799a9a89c891dcdfea98a9f551\": container with ID starting with 23c25ac730a8f54446d8c338499397e5dbcbb6799a9a89c891dcdfea98a9f551 not found: ID does not exist" Nov 04 05:28:17.538349 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.538328 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-trusted-ca\") pod \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " Nov 04 05:28:17.538465 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.538366 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-kube-api-access-8prxh\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-kube-api-access-8prxh\") pod \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " Nov 04 05:28:17.538465 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.538429 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-bound-sa-token\") pod \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " Nov 04 05:28:17.538581 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.538470 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-ca-trust-extracted\") pod \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " Nov 04 05:28:17.538633 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.538574 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\") pod \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " Nov 04 05:28:17.538780 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.538683 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-certificates\") pod \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " Nov 04 05:28:17.538780 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.538723 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-installation-pull-secrets\") pod \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " Nov 04 05:28:17.538780 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.538749 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-image-registry-private-configuration\") pod \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\" (UID: \"aa931f6e-a107-4ca3-bb54-e89a91d0f67a\") " Nov 04 05:28:17.538951 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.538781 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-trusted-ca" pod "aa931f6e-a107-4ca3-bb54-e89a91d0f67a" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:17.539073 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.539051 4859 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-trusted-ca\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:17.539173 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.539150 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-certificates" pod "aa931f6e-a107-4ca3-bb54-e89a91d0f67a" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:17.541092 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.541061 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-kube-api-access-8prxh" pod "aa931f6e-a107-4ca3-bb54-e89a91d0f67a" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a"). InnerVolumeSpecName "kube-api-access-8prxh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:28:17.541203 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.541145 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-bound-sa-token" pod "aa931f6e-a107-4ca3-bb54-e89a91d0f67a" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:28:17.541203 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.541151 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls" pod "aa931f6e-a107-4ca3-bb54-e89a91d0f67a" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:28:17.541551 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.541528 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-image-registry-private-configuration" pod "aa931f6e-a107-4ca3-bb54-e89a91d0f67a" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:17.542082 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.542057 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-installation-pull-secrets" pod "aa931f6e-a107-4ca3-bb54-e89a91d0f67a" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:17.546046 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.546022 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-ca-trust-extracted" pod "aa931f6e-a107-4ca3-bb54-e89a91d0f67a" (UID: "aa931f6e-a107-4ca3-bb54-e89a91d0f67a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:28:17.640071 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.640035 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8prxh\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-kube-api-access-8prxh\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:17.640071 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.640066 4859 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-bound-sa-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:17.640282 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.640082 4859 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-ca-trust-extracted\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:17.640282 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.640095 4859 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-tls\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:17.640282 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.640108 4859 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-registry-certificates\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:17.640282 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.640120 4859 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-installation-pull-secrets\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:17.640282 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.640135 4859 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aa931f6e-a107-4ca3-bb54-e89a91d0f67a-image-registry-private-configuration\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:17.903803 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.903766 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8f7d8c7c6-q48ph"] Nov 04 05:28:17.909506 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:17.909476 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8f7d8c7c6-q48ph"] Nov 04 05:28:18.113981 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:18.113908 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa931f6e-a107-4ca3-bb54-e89a91d0f67a" path="/var/lib/kubelet/pods/aa931f6e-a107-4ca3-bb54-e89a91d0f67a/volumes" Nov 04 05:28:18.472874 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:18.472838 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-29hbl" event={"ID":"b3ff1fa1-e55c-4a4f-8e3a-fcd0d341fc04","Type":"ContainerStarted","Data":"a5179a3eb04ac5aded426e5055397b98dd2056584ba83f149b1897b836ac9925"} Nov 04 05:28:22.488928 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:22.488895 4859 generic.go:356] "Generic (PLEG): container finished" podID="d26b56ed-c6e3-4321-8044-d29e5fe081c8" containerID="737ff35e608128b78aee1b9c3e4f24c4adc55e252a075ccaac9529b7b60add89" exitCode=0 Nov 04 05:28:22.489344 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:22.488968 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" event={"ID":"d26b56ed-c6e3-4321-8044-d29e5fe081c8","Type":"ContainerDied","Data":"737ff35e608128b78aee1b9c3e4f24c4adc55e252a075ccaac9529b7b60add89"} Nov 04 05:28:22.489344 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:22.489289 4859 scope.go:122] "RemoveContainer" containerID="737ff35e608128b78aee1b9c3e4f24c4adc55e252a075ccaac9529b7b60add89" Nov 04 05:28:23.494266 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:23.494226 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kqsd9" event={"ID":"d26b56ed-c6e3-4321-8044-d29e5fe081c8","Type":"ContainerStarted","Data":"aa3a33609a7d100df8dee5f7e9c4e4c3e7e1f458d6be4771f6cb6e5eaa79b07e"} Nov 04 05:28:26.507220 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:26.507184 4859 generic.go:356] "Generic (PLEG): container finished" podID="90fb287e-4af9-4e0b-919d-00713b37605c" containerID="1550427bdc24b8db1c10f40d5128980f9983c7bddecbcbae23bb1cf517a7a449" exitCode=0 Nov 04 05:28:26.507722 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:26.507225 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-b24ft" event={"ID":"90fb287e-4af9-4e0b-919d-00713b37605c","Type":"ContainerDied","Data":"1550427bdc24b8db1c10f40d5128980f9983c7bddecbcbae23bb1cf517a7a449"} Nov 04 05:28:26.507722 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:26.507602 4859 scope.go:122] "RemoveContainer" containerID="1550427bdc24b8db1c10f40d5128980f9983c7bddecbcbae23bb1cf517a7a449" Nov 04 05:28:27.513420 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:27.513383 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-b24ft" event={"ID":"90fb287e-4af9-4e0b-919d-00713b37605c","Type":"ContainerStarted","Data":"1297f64423add8c6110ac8347c64d9645a28771a9be3151dad48b436b12dae57"} Nov 04 05:28:28.340092 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:28:28.340050 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:28:51.979934 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:51.979898 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:28:51.980398 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:51.980185 4859 cpu_manager.go:414] "containerMap: removing container" podUID="00a06b1e-791a-4652-823c-b73672b7e4b2" containerName="registry" Nov 04 05:28:51.980398 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:51.980197 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00a06b1e-791a-4652-823c-b73672b7e4b2" containerName="registry" Nov 04 05:28:51.980398 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:51.980221 4859 cpu_manager.go:414] "containerMap: removing container" podUID="aa931f6e-a107-4ca3-bb54-e89a91d0f67a" containerName="registry" Nov 04 05:28:51.980398 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:51.980226 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa931f6e-a107-4ca3-bb54-e89a91d0f67a" containerName="registry" Nov 04 05:28:51.980398 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:51.980282 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa931f6e-a107-4ca3-bb54-e89a91d0f67a" containerName="registry" Nov 04 05:28:51.980398 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:51.980291 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="00a06b1e-791a-4652-823c-b73672b7e4b2" containerName="registry" Nov 04 05:28:51.983811 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:51.983794 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.029940 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.029909 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:28:52.037144 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.037122 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Nov 04 05:28:52.037330 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.037315 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Nov 04 05:28:52.058078 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.058055 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Nov 04 05:28:52.058189 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.058093 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Nov 04 05:28:52.058189 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.058104 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Nov 04 05:28:52.058189 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.058118 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Nov 04 05:28:52.058189 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.058147 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Nov 04 05:28:52.058189 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.058160 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Nov 04 05:28:52.058400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.058345 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dvjld\"" Nov 04 05:28:52.113159 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.113120 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ghwn\" (UniqueName: \"kubernetes.io/projected/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-kube-api-access-8ghwn\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.113533 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.113185 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.113533 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.113216 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.113533 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.113263 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-web-config\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.113533 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.113304 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.113533 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.113334 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-config-out\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.113533 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.113390 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.113533 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.113419 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-config-volume\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.113533 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.113447 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.113533 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.113480 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.113533 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.113513 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.114011 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.113543 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.114011 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.113567 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.138469 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.138437 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Nov 04 05:28:52.214347 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.214320 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.214484 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.214358 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-config-volume\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.214484 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.214380 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.214484 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.214409 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.214484 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.214435 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.214709 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.214480 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.214709 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.214521 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.214709 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.214563 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ghwn\" (UniqueName: \"kubernetes.io/projected/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-kube-api-access-8ghwn\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.214709 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.214597 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.214709 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.214619 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.214709 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.214696 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-web-config\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.215026 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.214733 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.215026 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.214751 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-config-out\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.216188 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.215502 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.216188 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.215570 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.216188 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.215854 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.217601 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.217555 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.217987 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.217961 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.218121 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.217999 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-config-out\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.218121 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.218101 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.218231 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.218116 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-config-volume\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.218231 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.218142 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.218515 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.218487 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.218796 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.218779 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.219556 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.219537 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-web-config\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.278105 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.278029 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ghwn\" (UniqueName: \"kubernetes.io/projected/bcbdb63b-fd44-4aaf-9dfe-769850a938ea-kube-api-access-8ghwn\") pod \"alertmanager-main-0\" (UID: \"bcbdb63b-fd44-4aaf-9dfe-769850a938ea\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.308157 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.308124 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:52.734883 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:28:52.734854 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcbdb63b_fd44_4aaf_9dfe_769850a938ea.slice/crio-749d603ea2cdfce9d875d764ee0d2822f6dcf773fcfeb6acee465942729ef67a WatchSource:0}: Error finding container 749d603ea2cdfce9d875d764ee0d2822f6dcf773fcfeb6acee465942729ef67a: Status 404 returned error can't find the container with id 749d603ea2cdfce9d875d764ee0d2822f6dcf773fcfeb6acee465942729ef67a Nov 04 05:28:52.736497 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:52.736472 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:28:53.610925 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:53.610893 4859 generic.go:356] "Generic (PLEG): container finished" podID="bcbdb63b-fd44-4aaf-9dfe-769850a938ea" containerID="f619f461ffa949a95200c9472e28c2ed4462d7d29691c81166004634a589328c" exitCode=0 Nov 04 05:28:53.611318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:53.610964 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bcbdb63b-fd44-4aaf-9dfe-769850a938ea","Type":"ContainerDied","Data":"f619f461ffa949a95200c9472e28c2ed4462d7d29691c81166004634a589328c"} Nov 04 05:28:53.611318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:53.611015 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bcbdb63b-fd44-4aaf-9dfe-769850a938ea","Type":"ContainerStarted","Data":"749d603ea2cdfce9d875d764ee0d2822f6dcf773fcfeb6acee465942729ef67a"} Nov 04 05:28:56.626781 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:56.626746 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bcbdb63b-fd44-4aaf-9dfe-769850a938ea","Type":"ContainerStarted","Data":"63a25f35b533e67ba88107ad750127f58a1f26c9214d876fd9be95b2fc201aaf"} Nov 04 05:28:56.627155 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:56.626814 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bcbdb63b-fd44-4aaf-9dfe-769850a938ea","Type":"ContainerStarted","Data":"25097c02f76ce40a91d7ce6cde45f6553295e5bf1a6b0048038fa59e453382e7"} Nov 04 05:28:56.627155 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:56.626840 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bcbdb63b-fd44-4aaf-9dfe-769850a938ea","Type":"ContainerStarted","Data":"a927dddc71c5e1cdddca8a81256d8f3d8620ccf6f1040b52fc95a49de822a01f"} Nov 04 05:28:56.627155 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:56.626863 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bcbdb63b-fd44-4aaf-9dfe-769850a938ea","Type":"ContainerStarted","Data":"2b7109a551a778a4218b28b039b227ecc8a74a029edd924152baaf99436b5cbd"} Nov 04 05:28:56.627155 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:56.626886 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bcbdb63b-fd44-4aaf-9dfe-769850a938ea","Type":"ContainerStarted","Data":"1b7fa465aa598448f84f879d0473c28e23d1efe7249fa05fec8ace3366130ce5"} Nov 04 05:28:57.634059 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:57.634020 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bcbdb63b-fd44-4aaf-9dfe-769850a938ea","Type":"ContainerStarted","Data":"0bd5d967ab5249fa7a58ca5e59c2d9faffee0fef5d80b2c6c36cb2ad26fbef84"} Nov 04 05:28:57.913455 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:28:57.913404 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.339255887 podStartE2EDuration="6.91339161s" totalImagesPullingTime="3.56221248s" totalInitContainerRuntime="11.923243ms" isStatefulPod=false podCreationTimestamp="2026-11-04 05:28:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:57.863106697 +0000 UTC m=+107.864742433" watchObservedRunningTime="2026-11-04 05:28:57.91339161 +0000 UTC m=+107.915027314" Nov 04 05:28:58.340145 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:28:58.340065 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:29:28.340505 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:29:28.340469 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:29:58.341859 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:29:58.341820 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:30:28.342152 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:30:28.342111 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:30:58.340589 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:30:58.340497 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:31:07.724687 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.724625 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq"] Nov 04 05:31:07.728078 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.728059 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.784295 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.784256 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Nov 04 05:31:07.784295 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.784252 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Nov 04 05:31:07.784295 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.784297 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Nov 04 05:31:07.784547 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.784295 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Nov 04 05:31:07.784547 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.784350 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-v5pxw\"" Nov 04 05:31:07.833932 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.833891 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.834102 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.833938 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.834102 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.834019 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.834102 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.834083 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.834206 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.834129 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.834206 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.834152 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmr76\" (UniqueName: \"kubernetes.io/projected/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-kube-api-access-hmr76\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.834206 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.834172 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.835657 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.835635 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq"] Nov 04 05:31:07.935475 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.935428 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.935475 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.935479 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.935783 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.935516 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmr76\" (UniqueName: \"kubernetes.io/projected/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-kube-api-access-hmr76\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.935783 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.935544 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.935783 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.935593 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.935783 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.935627 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.935783 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.935650 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.936467 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.936437 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.938089 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.938060 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.938189 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.938101 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.938189 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.938122 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.938263 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.938193 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.997503 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.997430 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:07.997659 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:07.997632 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmr76\" (UniqueName: \"kubernetes.io/projected/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-kube-api-access-hmr76\") pod \"istiod-openshift-gateway-7876bfcb74-9cbtq\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:08.039759 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:08.039707 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:08.473892 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:08.473823 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq"] Nov 04 05:31:08.479471 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:31:08.479441 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e48bce_faa6_4ef0_a3e8_c5a5473af8b5.slice/crio-573db515ec357fe3b8f03109f3d558c2951cf87d0d3cfb70b469118132d8a206 WatchSource:0}: Error finding container 573db515ec357fe3b8f03109f3d558c2951cf87d0d3cfb70b469118132d8a206: Status 404 returned error can't find the container with id 573db515ec357fe3b8f03109f3d558c2951cf87d0d3cfb70b469118132d8a206 Nov 04 05:31:09.151082 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:09.151049 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" event={"ID":"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5","Type":"ContainerStarted","Data":"573db515ec357fe3b8f03109f3d558c2951cf87d0d3cfb70b469118132d8a206"} Nov 04 05:31:13.614264 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:13.614223 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:31:13.614641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:13.614307 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:31:14.172796 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:14.172761 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" event={"ID":"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5","Type":"ContainerStarted","Data":"1115925fa75d69172c372cbc8482f2e885662849831bd0285d764416e1dac7f6"} Nov 04 05:31:14.172971 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:14.172938 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:14.174624 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:14.174597 4859 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-9cbtq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:31:14.174789 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:14.174685 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" podUID="f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:31:14.382792 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:14.382730 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" podStartSLOduration=2.250910936 podStartE2EDuration="7.382715458s" totalImagesPullingTime="5.131804522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:31:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:31:14.344696816 +0000 UTC m=+244.346332532" watchObservedRunningTime="2026-11-04 05:31:14.382715458 +0000 UTC m=+244.384351182" Nov 04 05:31:15.177591 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:15.177554 4859 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-9cbtq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:31:15.178085 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:15.177614 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" podUID="f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:31:18.177786 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:18.177751 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:31:28.341079 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:31:28.341040 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:31:58.243532 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.243486 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr"] Nov 04 05:31:58.247010 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.246985 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.326132 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.326095 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" Nov 04 05:31:58.326293 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.326153 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" Nov 04 05:31:58.328871 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.328847 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" Nov 04 05:31:58.329050 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.329030 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\"" Nov 04 05:31:58.329120 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.329065 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" Nov 04 05:31:58.329120 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.329064 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-ssrcr\"" Nov 04 05:31:58.342650 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.342621 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/33f5fdba-b9af-4082-819c-2b26ca04d1d0-manifests\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.342883 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.342699 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/33f5fdba-b9af-4082-819c-2b26ca04d1d0-metrics-cert\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.342883 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.342740 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33f5fdba-b9af-4082-819c-2b26ca04d1d0-tmp\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.342883 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.342857 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58pfg\" (UniqueName: \"kubernetes.io/projected/33f5fdba-b9af-4082-819c-2b26ca04d1d0-kube-api-access-58pfg\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.343049 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.342886 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33f5fdba-b9af-4082-819c-2b26ca04d1d0-webhook-cert\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.343049 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.343022 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/33f5fdba-b9af-4082-819c-2b26ca04d1d0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.345241 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:31:58.345212 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:31:58.381562 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.381522 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr"] Nov 04 05:31:58.443377 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.443338 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/33f5fdba-b9af-4082-819c-2b26ca04d1d0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.443567 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.443391 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/33f5fdba-b9af-4082-819c-2b26ca04d1d0-manifests\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.443567 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.443411 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/33f5fdba-b9af-4082-819c-2b26ca04d1d0-metrics-cert\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.443567 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.443427 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33f5fdba-b9af-4082-819c-2b26ca04d1d0-tmp\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.443567 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.443445 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58pfg\" (UniqueName: \"kubernetes.io/projected/33f5fdba-b9af-4082-819c-2b26ca04d1d0-kube-api-access-58pfg\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.443567 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.443474 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33f5fdba-b9af-4082-819c-2b26ca04d1d0-webhook-cert\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.443943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.443918 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/33f5fdba-b9af-4082-819c-2b26ca04d1d0-manifests\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.446007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.445981 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33f5fdba-b9af-4082-819c-2b26ca04d1d0-tmp\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.446154 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.446137 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/33f5fdba-b9af-4082-819c-2b26ca04d1d0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.446200 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.446161 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/33f5fdba-b9af-4082-819c-2b26ca04d1d0-metrics-cert\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.446285 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.446270 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33f5fdba-b9af-4082-819c-2b26ca04d1d0-webhook-cert\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.518140 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.518070 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58pfg\" (UniqueName: \"kubernetes.io/projected/33f5fdba-b9af-4082-819c-2b26ca04d1d0-kube-api-access-58pfg\") pod \"opendatahub-operator-controller-manager-c98788444-gfgvr\" (UID: \"33f5fdba-b9af-4082-819c-2b26ca04d1d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:58.562919 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:58.562880 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:31:59.050228 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:31:59.050200 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33f5fdba_b9af_4082_819c_2b26ca04d1d0.slice/crio-70fbb966865bc6d097c97be0626df0c1e6e1cb8ec723fe508b767cd8c3101cce WatchSource:0}: Error finding container 70fbb966865bc6d097c97be0626df0c1e6e1cb8ec723fe508b767cd8c3101cce: Status 404 returned error can't find the container with id 70fbb966865bc6d097c97be0626df0c1e6e1cb8ec723fe508b767cd8c3101cce Nov 04 05:31:59.050376 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:59.050246 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr"] Nov 04 05:31:59.352288 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:31:59.352199 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" event={"ID":"33f5fdba-b9af-4082-819c-2b26ca04d1d0","Type":"ContainerStarted","Data":"70fbb966865bc6d097c97be0626df0c1e6e1cb8ec723fe508b767cd8c3101cce"} Nov 04 05:32:03.372150 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:03.372109 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" event={"ID":"33f5fdba-b9af-4082-819c-2b26ca04d1d0","Type":"ContainerStarted","Data":"c954b8b4b18e6b3274f75631f851b7f8739cedcde00e16baf4dbd15994d1381a"} Nov 04 05:32:04.378009 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:04.377971 4859 generic.go:356] "Generic (PLEG): container finished" podID="33f5fdba-b9af-4082-819c-2b26ca04d1d0" containerID="c954b8b4b18e6b3274f75631f851b7f8739cedcde00e16baf4dbd15994d1381a" exitCode=0 Nov 04 05:32:04.378389 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:04.378056 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" event={"ID":"33f5fdba-b9af-4082-819c-2b26ca04d1d0","Type":"ContainerDied","Data":"c954b8b4b18e6b3274f75631f851b7f8739cedcde00e16baf4dbd15994d1381a"} Nov 04 05:32:06.388196 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:06.388149 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" event={"ID":"33f5fdba-b9af-4082-819c-2b26ca04d1d0","Type":"ContainerStarted","Data":"652d7abe4ef8e195450e2ead9f41d5255573a81c5a946075cadc46b9d9a92762"} Nov 04 05:32:06.388583 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:06.388241 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:32:06.698841 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:06.698787 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" podStartSLOduration=3.476540873 podStartE2EDuration="8.698771122s" totalImagesPullingTime="4.8075613s" totalInitContainerRuntime="414.668949ms" isStatefulPod=false podCreationTimestamp="2026-11-04 05:31:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:06.643338298 +0000 UTC m=+296.644974023" watchObservedRunningTime="2026-11-04 05:32:06.698771122 +0000 UTC m=+296.700406846" Nov 04 05:32:10.125442 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:10.125412 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/2.log" Nov 04 05:32:10.126319 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:10.126302 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/2.log" Nov 04 05:32:17.394952 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:17.394918 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-gfgvr" Nov 04 05:32:28.341760 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:32:28.341716 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:32:34.171463 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.171424 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t"] Nov 04 05:32:34.175022 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.175004 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t" Nov 04 05:32:34.252811 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.252773 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccec3278-ee58-40cc-a076-6380de7e1f72-tls-certs\") pod \"kube-auth-proxy-5c6f4696b9-jdr4t\" (UID: \"ccec3278-ee58-40cc-a076-6380de7e1f72\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t" Nov 04 05:32:34.252981 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.252818 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccec3278-ee58-40cc-a076-6380de7e1f72-tmp\") pod \"kube-auth-proxy-5c6f4696b9-jdr4t\" (UID: \"ccec3278-ee58-40cc-a076-6380de7e1f72\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t" Nov 04 05:32:34.253029 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.252987 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8fpm\" (UniqueName: \"kubernetes.io/projected/ccec3278-ee58-40cc-a076-6380de7e1f72-kube-api-access-x8fpm\") pod \"kube-auth-proxy-5c6f4696b9-jdr4t\" (UID: \"ccec3278-ee58-40cc-a076-6380de7e1f72\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t" Nov 04 05:32:34.286183 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.286149 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-x2vv4\"" Nov 04 05:32:34.286183 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.286143 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Nov 04 05:32:34.286412 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.286224 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Nov 04 05:32:34.287353 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.287328 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t"] Nov 04 05:32:34.353808 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.353768 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccec3278-ee58-40cc-a076-6380de7e1f72-tls-certs\") pod \"kube-auth-proxy-5c6f4696b9-jdr4t\" (UID: \"ccec3278-ee58-40cc-a076-6380de7e1f72\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t" Nov 04 05:32:34.353808 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.353812 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccec3278-ee58-40cc-a076-6380de7e1f72-tmp\") pod \"kube-auth-proxy-5c6f4696b9-jdr4t\" (UID: \"ccec3278-ee58-40cc-a076-6380de7e1f72\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t" Nov 04 05:32:34.354040 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.353869 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8fpm\" (UniqueName: \"kubernetes.io/projected/ccec3278-ee58-40cc-a076-6380de7e1f72-kube-api-access-x8fpm\") pod \"kube-auth-proxy-5c6f4696b9-jdr4t\" (UID: \"ccec3278-ee58-40cc-a076-6380de7e1f72\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t" Nov 04 05:32:34.356285 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.356252 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccec3278-ee58-40cc-a076-6380de7e1f72-tmp\") pod \"kube-auth-proxy-5c6f4696b9-jdr4t\" (UID: \"ccec3278-ee58-40cc-a076-6380de7e1f72\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t" Nov 04 05:32:34.356413 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.356399 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccec3278-ee58-40cc-a076-6380de7e1f72-tls-certs\") pod \"kube-auth-proxy-5c6f4696b9-jdr4t\" (UID: \"ccec3278-ee58-40cc-a076-6380de7e1f72\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t" Nov 04 05:32:34.419180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.419138 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8fpm\" (UniqueName: \"kubernetes.io/projected/ccec3278-ee58-40cc-a076-6380de7e1f72-kube-api-access-x8fpm\") pod \"kube-auth-proxy-5c6f4696b9-jdr4t\" (UID: \"ccec3278-ee58-40cc-a076-6380de7e1f72\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t" Nov 04 05:32:34.487682 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.487572 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t" Nov 04 05:32:34.892540 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.892459 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t"] Nov 04 05:32:34.897565 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:32:34.897539 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccec3278_ee58_40cc_a076_6380de7e1f72.slice/crio-130f505af4d1c91b961e1ff43f389f4781202f068c23d8bcdf83bd9ac20f4f5e WatchSource:0}: Error finding container 130f505af4d1c91b961e1ff43f389f4781202f068c23d8bcdf83bd9ac20f4f5e: Status 404 returned error can't find the container with id 130f505af4d1c91b961e1ff43f389f4781202f068c23d8bcdf83bd9ac20f4f5e Nov 04 05:32:34.899433 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:34.899418 4859 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:32:35.513210 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:35.513176 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t" event={"ID":"ccec3278-ee58-40cc-a076-6380de7e1f72","Type":"ContainerStarted","Data":"130f505af4d1c91b961e1ff43f389f4781202f068c23d8bcdf83bd9ac20f4f5e"} Nov 04 05:32:39.534579 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:39.534538 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t" event={"ID":"ccec3278-ee58-40cc-a076-6380de7e1f72","Type":"ContainerStarted","Data":"4cc88e951372f865404f50be131c5a3d56b2d548f0a5073a0b5d20ffe22e4a8f"} Nov 04 05:32:39.661563 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:39.661504 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-jdr4t" podStartSLOduration=1.865998131 podStartE2EDuration="5.661491228s" totalImagesPullingTime="3.795493097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:39.660721201 +0000 UTC m=+329.662356926" watchObservedRunningTime="2026-11-04 05:32:39.661491228 +0000 UTC m=+329.663126988" Nov 04 05:32:47.358750 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.358714 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk"] Nov 04 05:32:47.362193 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.362172 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" Nov 04 05:32:47.420951 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.420916 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Nov 04 05:32:47.421131 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.420988 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Nov 04 05:32:47.421131 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.421058 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-bs9p6\"" Nov 04 05:32:47.470787 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.470754 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sx24\" (UniqueName: \"kubernetes.io/projected/2a69b293-4aa6-4f66-b471-477cf841ce67-kube-api-access-7sx24\") pod \"kserve-module-controller-manager-7566cf5fc8-m8srk\" (UID: \"2a69b293-4aa6-4f66-b471-477cf841ce67\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" Nov 04 05:32:47.470972 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.470803 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2a69b293-4aa6-4f66-b471-477cf841ce67-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-m8srk\" (UID: \"2a69b293-4aa6-4f66-b471-477cf841ce67\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" Nov 04 05:32:47.470972 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.470898 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2a69b293-4aa6-4f66-b471-477cf841ce67-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-m8srk\" (UID: \"2a69b293-4aa6-4f66-b471-477cf841ce67\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" Nov 04 05:32:47.470972 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.470933 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a69b293-4aa6-4f66-b471-477cf841ce67-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-m8srk\" (UID: \"2a69b293-4aa6-4f66-b471-477cf841ce67\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" Nov 04 05:32:47.567650 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.567617 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\"" Nov 04 05:32:47.568770 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.568751 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk"] Nov 04 05:32:47.571544 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.571524 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sx24\" (UniqueName: \"kubernetes.io/projected/2a69b293-4aa6-4f66-b471-477cf841ce67-kube-api-access-7sx24\") pod \"kserve-module-controller-manager-7566cf5fc8-m8srk\" (UID: \"2a69b293-4aa6-4f66-b471-477cf841ce67\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" Nov 04 05:32:47.571596 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.571564 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2a69b293-4aa6-4f66-b471-477cf841ce67-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-m8srk\" (UID: \"2a69b293-4aa6-4f66-b471-477cf841ce67\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" Nov 04 05:32:47.571705 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.571684 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2a69b293-4aa6-4f66-b471-477cf841ce67-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-m8srk\" (UID: \"2a69b293-4aa6-4f66-b471-477cf841ce67\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" Nov 04 05:32:47.571761 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.571746 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a69b293-4aa6-4f66-b471-477cf841ce67-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-m8srk\" (UID: \"2a69b293-4aa6-4f66-b471-477cf841ce67\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" Nov 04 05:32:47.572140 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.572125 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2a69b293-4aa6-4f66-b471-477cf841ce67-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-m8srk\" (UID: \"2a69b293-4aa6-4f66-b471-477cf841ce67\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" Nov 04 05:32:47.572220 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.572199 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a69b293-4aa6-4f66-b471-477cf841ce67-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-m8srk\" (UID: \"2a69b293-4aa6-4f66-b471-477cf841ce67\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" Nov 04 05:32:47.572535 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.572520 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2a69b293-4aa6-4f66-b471-477cf841ce67-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-m8srk\" (UID: \"2a69b293-4aa6-4f66-b471-477cf841ce67\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" Nov 04 05:32:47.642571 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.642490 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sx24\" (UniqueName: \"kubernetes.io/projected/2a69b293-4aa6-4f66-b471-477cf841ce67-kube-api-access-7sx24\") pod \"kserve-module-controller-manager-7566cf5fc8-m8srk\" (UID: \"2a69b293-4aa6-4f66-b471-477cf841ce67\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" Nov 04 05:32:47.674042 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:47.674007 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" Nov 04 05:32:48.107616 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:48.107581 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk"] Nov 04 05:32:48.113055 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:32:48.113025 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a69b293_4aa6_4f66_b471_477cf841ce67.slice/crio-5137d44affd64a3e2b954327b30a0cdcd950a85bca40e8e27fce27d25959f7dc WatchSource:0}: Error finding container 5137d44affd64a3e2b954327b30a0cdcd950a85bca40e8e27fce27d25959f7dc: Status 404 returned error can't find the container with id 5137d44affd64a3e2b954327b30a0cdcd950a85bca40e8e27fce27d25959f7dc Nov 04 05:32:48.571640 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:48.571598 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" event={"ID":"2a69b293-4aa6-4f66-b471-477cf841ce67","Type":"ContainerStarted","Data":"5137d44affd64a3e2b954327b30a0cdcd950a85bca40e8e27fce27d25959f7dc"} Nov 04 05:32:51.586070 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:51.586031 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" event={"ID":"2a69b293-4aa6-4f66-b471-477cf841ce67","Type":"ContainerStarted","Data":"f5870f802b6e75647bb7373a4cc3858a8716e46735aff8bbc49881073a5b1a3a"} Nov 04 05:32:51.586472 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:51.586109 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" Nov 04 05:32:51.767795 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:32:51.767735 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" podStartSLOduration=1.764316687 podStartE2EDuration="4.767722354s" totalImagesPullingTime="3.003405667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:51.767271118 +0000 UTC m=+341.768906844" watchObservedRunningTime="2026-11-04 05:32:51.767722354 +0000 UTC m=+341.769358075" Nov 04 05:32:58.341302 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:32:58.341259 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:33:12.591656 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:12.591621 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-m8srk" Nov 04 05:33:28.341592 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:33:28.341553 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:33:35.379816 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.379765 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4"] Nov 04 05:33:35.386558 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.386531 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:35.441098 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.441049 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4"] Nov 04 05:33:35.493748 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.493701 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\"" Nov 04 05:33:35.493748 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.493730 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\"" Nov 04 05:33:35.493748 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.493709 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\"" Nov 04 05:33:35.494071 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.493794 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-49glt\"" Nov 04 05:33:35.502148 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.502117 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b132e4f-1cda-4c76-820a-b81feb73ad40-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cjkv4\" (UID: \"1b132e4f-1cda-4c76-820a-b81feb73ad40\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:35.502148 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.502153 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b132e4f-1cda-4c76-820a-b81feb73ad40-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cjkv4\" (UID: \"1b132e4f-1cda-4c76-820a-b81feb73ad40\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:35.502374 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.502175 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9ddb\" (UniqueName: \"kubernetes.io/projected/1b132e4f-1cda-4c76-820a-b81feb73ad40-kube-api-access-f9ddb\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cjkv4\" (UID: \"1b132e4f-1cda-4c76-820a-b81feb73ad40\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:35.502432 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.502378 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/1b132e4f-1cda-4c76-820a-b81feb73ad40-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cjkv4\" (UID: \"1b132e4f-1cda-4c76-820a-b81feb73ad40\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:35.603174 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.603126 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b132e4f-1cda-4c76-820a-b81feb73ad40-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cjkv4\" (UID: \"1b132e4f-1cda-4c76-820a-b81feb73ad40\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:35.603174 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.603177 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9ddb\" (UniqueName: \"kubernetes.io/projected/1b132e4f-1cda-4c76-820a-b81feb73ad40-kube-api-access-f9ddb\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cjkv4\" (UID: \"1b132e4f-1cda-4c76-820a-b81feb73ad40\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:35.603381 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.603258 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/1b132e4f-1cda-4c76-820a-b81feb73ad40-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cjkv4\" (UID: \"1b132e4f-1cda-4c76-820a-b81feb73ad40\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:35.603381 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.603279 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b132e4f-1cda-4c76-820a-b81feb73ad40-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cjkv4\" (UID: \"1b132e4f-1cda-4c76-820a-b81feb73ad40\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:35.603381 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:33:35.603373 4859 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found Nov 04 05:33:35.603479 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:33:35.603380 4859 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found Nov 04 05:33:35.603479 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:33:35.603440 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b132e4f-1cda-4c76-820a-b81feb73ad40-tls-cert podName:1b132e4f-1cda-4c76-820a-b81feb73ad40 nodeName:}" failed. No retries permitted until 2026-11-04 05:33:36.103423099 +0000 UTC m=+386.105058802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/1b132e4f-1cda-4c76-820a-b81feb73ad40-tls-cert") pod "usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" (UID: "1b132e4f-1cda-4c76-820a-b81feb73ad40") : secret "usage-logs-tenancy-proxy-tls" not found Nov 04 05:33:35.603479 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:33:35.603455 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b132e4f-1cda-4c76-820a-b81feb73ad40-proxy-source podName:1b132e4f-1cda-4c76-820a-b81feb73ad40 nodeName:}" failed. No retries permitted until 2026-11-04 05:33:36.103448314 +0000 UTC m=+386.105084016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/1b132e4f-1cda-4c76-820a-b81feb73ad40-proxy-source") pod "usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" (UID: "1b132e4f-1cda-4c76-820a-b81feb73ad40") : configmap "usage-tenancy-proxy-source" not found Nov 04 05:33:35.603915 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.603895 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b132e4f-1cda-4c76-820a-b81feb73ad40-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cjkv4\" (UID: \"1b132e4f-1cda-4c76-820a-b81feb73ad40\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:35.670799 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:35.670773 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9ddb\" (UniqueName: \"kubernetes.io/projected/1b132e4f-1cda-4c76-820a-b81feb73ad40-kube-api-access-f9ddb\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cjkv4\" (UID: \"1b132e4f-1cda-4c76-820a-b81feb73ad40\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:36.108752 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:36.108628 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/1b132e4f-1cda-4c76-820a-b81feb73ad40-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cjkv4\" (UID: \"1b132e4f-1cda-4c76-820a-b81feb73ad40\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:36.108752 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:36.108710 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b132e4f-1cda-4c76-820a-b81feb73ad40-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cjkv4\" (UID: \"1b132e4f-1cda-4c76-820a-b81feb73ad40\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:36.109499 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:36.109471 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/1b132e4f-1cda-4c76-820a-b81feb73ad40-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cjkv4\" (UID: \"1b132e4f-1cda-4c76-820a-b81feb73ad40\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:36.111199 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:36.111168 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b132e4f-1cda-4c76-820a-b81feb73ad40-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cjkv4\" (UID: \"1b132e4f-1cda-4c76-820a-b81feb73ad40\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:36.299907 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:36.299867 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:36.679398 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:36.679359 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4"] Nov 04 05:33:36.684264 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:33:36.684236 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b132e4f_1cda_4c76_820a_b81feb73ad40.slice/crio-34341fad0c032e17a72ac16c124a1e2a04a0b2a57640b203f17c97a06b18ee46 WatchSource:0}: Error finding container 34341fad0c032e17a72ac16c124a1e2a04a0b2a57640b203f17c97a06b18ee46: Status 404 returned error can't find the container with id 34341fad0c032e17a72ac16c124a1e2a04a0b2a57640b203f17c97a06b18ee46 Nov 04 05:33:36.789841 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:36.789803 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" event={"ID":"1b132e4f-1cda-4c76-820a-b81feb73ad40","Type":"ContainerStarted","Data":"34341fad0c032e17a72ac16c124a1e2a04a0b2a57640b203f17c97a06b18ee46"} Nov 04 05:33:49.488168 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.488122 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr"] Nov 04 05:33:49.493942 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.493919 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.532022 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.531976 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.532214 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.532031 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.532214 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.532181 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.532347 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.532234 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.532347 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.532271 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpp2p\" (UniqueName: \"kubernetes.io/projected/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-kube-api-access-mpp2p\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.532430 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.532353 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.532430 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.532400 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.606591 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.606556 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr"] Nov 04 05:33:49.632997 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.632963 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.633172 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.633000 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.633172 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.633061 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.633172 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.633090 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.633172 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.633120 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpp2p\" (UniqueName: \"kubernetes.io/projected/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-kube-api-access-mpp2p\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.633172 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.633157 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.633421 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.633181 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.634054 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.634019 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.635627 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.635602 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.635770 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.635731 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.635840 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.635792 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.635962 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.635942 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.699337 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.699300 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.699542 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.699513 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpp2p\" (UniqueName: \"kubernetes.io/projected/e371ee7d-fd98-4f76-b0ca-01dc7f636ba3-kube-api-access-mpp2p\") pod \"istiod-openshift-gateway-7ddd9cfdc8-z4xcr\" (UID: \"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:49.806898 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:49.806823 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:50.244809 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:50.244770 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr"] Nov 04 05:33:50.249837 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:33:50.249804 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode371ee7d_fd98_4f76_b0ca_01dc7f636ba3.slice/crio-0be54b54b66e7379898966d9cc6472511c2ac80980f64f99b3d4b3da7a9299c5 WatchSource:0}: Error finding container 0be54b54b66e7379898966d9cc6472511c2ac80980f64f99b3d4b3da7a9299c5: Status 404 returned error can't find the container with id 0be54b54b66e7379898966d9cc6472511c2ac80980f64f99b3d4b3da7a9299c5 Nov 04 05:33:50.251955 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:50.251921 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:33:50.252041 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:50.251991 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:33:50.862008 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:50.861966 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" event={"ID":"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3","Type":"ContainerStarted","Data":"ed8f6a55d837135dc97ac25fda8d4b45b131dc90133a5420aa08d7869b54f4ce"} Nov 04 05:33:50.862401 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:50.862027 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" event={"ID":"e371ee7d-fd98-4f76-b0ca-01dc7f636ba3","Type":"ContainerStarted","Data":"0be54b54b66e7379898966d9cc6472511c2ac80980f64f99b3d4b3da7a9299c5"} Nov 04 05:33:50.862401 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:50.862047 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:50.991862 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:50.991798 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" podStartSLOduration=1.991779948 podStartE2EDuration="1.991779948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:33:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:33:50.991441719 +0000 UTC m=+400.993077467" watchObservedRunningTime="2026-11-04 05:33:50.991779948 +0000 UTC m=+400.993415676" Nov 04 05:33:51.868327 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:51.868299 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z4xcr" Nov 04 05:33:52.115007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.114975 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq"] Nov 04 05:33:52.115235 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.115211 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" podUID="f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5" containerName="discovery" containerID="cri-o://1115925fa75d69172c372cbc8482f2e885662849831bd0285d764416e1dac7f6" gracePeriod=30 Nov 04 05:33:52.374436 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.374410 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:33:52.453430 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.453348 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-kube-api-access-hmr76\" (UniqueName: \"kubernetes.io/projected/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-kube-api-access-hmr76\") pod \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " Nov 04 05:33:52.453430 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.453402 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-kubeconfig\") pod \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " Nov 04 05:33:52.453430 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.453431 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-csr-ca-configmap\") pod \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " Nov 04 05:33:52.453698 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.453467 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-local-certs\") pod \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " Nov 04 05:33:52.453698 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.453491 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-cacerts\" (UniqueName: \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-cacerts\") pod \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " Nov 04 05:33:52.453698 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.453525 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-csr-dns-cert\") pod \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " Nov 04 05:33:52.453698 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.453570 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-token\" (UniqueName: \"kubernetes.io/projected/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-token\") pod \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\" (UID: \"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5\") " Nov 04 05:33:52.454215 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.454161 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-csr-ca-configmap" pod "f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5" (UID: "f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:33:52.456586 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.456536 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-csr-dns-cert" pod "f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5" (UID: "f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:33:52.456751 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.456627 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-token" pod "f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5" (UID: "f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:33:52.456751 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.456630 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-cacerts" pod "f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5" (UID: "f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:33:52.457007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.456828 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-kubeconfig" pod "f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5" (UID: "f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:33:52.457007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.456955 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-kube-api-access-hmr76" pod "f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5" (UID: "f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5"). InnerVolumeSpecName "kube-api-access-hmr76". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:33:52.457835 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.457798 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-local-certs" pod "f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5" (UID: "f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:33:52.554777 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.554735 4859 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-local-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:33:52.554777 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.554769 4859 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-cacerts\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:33:52.554777 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.554779 4859 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-csr-dns-cert\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:33:52.554777 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.554788 4859 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:33:52.555034 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.554797 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmr76\" (UniqueName: \"kubernetes.io/projected/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-kube-api-access-hmr76\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:33:52.555034 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.554806 4859 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-kubeconfig\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:33:52.555034 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.554816 4859 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5-istio-csr-ca-configmap\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:33:52.872371 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.872280 4859 generic.go:356] "Generic (PLEG): container finished" podID="f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5" containerID="1115925fa75d69172c372cbc8482f2e885662849831bd0285d764416e1dac7f6" exitCode=0 Nov 04 05:33:52.872371 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.872337 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" Nov 04 05:33:52.872371 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.872355 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" event={"ID":"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5","Type":"ContainerDied","Data":"1115925fa75d69172c372cbc8482f2e885662849831bd0285d764416e1dac7f6"} Nov 04 05:33:52.872914 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.872415 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq" event={"ID":"f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5","Type":"ContainerDied","Data":"573db515ec357fe3b8f03109f3d558c2951cf87d0d3cfb70b469118132d8a206"} Nov 04 05:33:52.872914 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.872443 4859 scope.go:122] "RemoveContainer" containerID="1115925fa75d69172c372cbc8482f2e885662849831bd0285d764416e1dac7f6" Nov 04 05:33:52.883412 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.883390 4859 scope.go:122] "RemoveContainer" containerID="1115925fa75d69172c372cbc8482f2e885662849831bd0285d764416e1dac7f6" Nov 04 05:33:52.883799 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:33:52.883767 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1115925fa75d69172c372cbc8482f2e885662849831bd0285d764416e1dac7f6\": container with ID starting with 1115925fa75d69172c372cbc8482f2e885662849831bd0285d764416e1dac7f6 not found: ID does not exist" containerID="1115925fa75d69172c372cbc8482f2e885662849831bd0285d764416e1dac7f6" Nov 04 05:33:52.883887 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:52.883810 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1115925fa75d69172c372cbc8482f2e885662849831bd0285d764416e1dac7f6"} err="failed to get container status \"1115925fa75d69172c372cbc8482f2e885662849831bd0285d764416e1dac7f6\": rpc error: code = NotFound desc = could not find container \"1115925fa75d69172c372cbc8482f2e885662849831bd0285d764416e1dac7f6\": container with ID starting with 1115925fa75d69172c372cbc8482f2e885662849831bd0285d764416e1dac7f6 not found: ID does not exist" Nov 04 05:33:53.050298 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:53.050260 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq"] Nov 04 05:33:53.104959 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:53.104918 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-9cbtq"] Nov 04 05:33:54.115175 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:54.115134 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5" path="/var/lib/kubelet/pods/f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5/volumes" Nov 04 05:33:55.513810 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.513774 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-w5msn"] Nov 04 05:33:55.514344 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.514325 4859 cpu_manager.go:414] "containerMap: removing container" podUID="f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5" containerName="discovery" Nov 04 05:33:55.514420 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.514347 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5" containerName="discovery" Nov 04 05:33:55.514470 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.514463 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1e48bce-faa6-4ef0-a3e8-c5a5473af8b5" containerName="discovery" Nov 04 05:33:55.540470 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.540436 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-w5msn" Nov 04 05:33:55.575428 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.575388 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-w5msn"] Nov 04 05:33:55.586064 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.586024 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wspsq\" (UniqueName: \"kubernetes.io/projected/8972ea5d-5467-4935-b341-d3fbd7750a21-kube-api-access-wspsq\") pod \"model-serving-api-75bd89fdfc-w5msn\" (UID: \"8972ea5d-5467-4935-b341-d3fbd7750a21\") " pod="opendatahub/model-serving-api-75bd89fdfc-w5msn" Nov 04 05:33:55.586211 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.586161 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8972ea5d-5467-4935-b341-d3fbd7750a21-tls-certs\") pod \"model-serving-api-75bd89fdfc-w5msn\" (UID: \"8972ea5d-5467-4935-b341-d3fbd7750a21\") " pod="opendatahub/model-serving-api-75bd89fdfc-w5msn" Nov 04 05:33:55.630060 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.630017 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-rv57b"] Nov 04 05:33:55.635247 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.635219 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-rv57b" Nov 04 05:33:55.652710 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.652681 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Nov 04 05:33:55.652873 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.652740 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-86zhq\"" Nov 04 05:33:55.687486 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.687443 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8972ea5d-5467-4935-b341-d3fbd7750a21-tls-certs\") pod \"model-serving-api-75bd89fdfc-w5msn\" (UID: \"8972ea5d-5467-4935-b341-d3fbd7750a21\") " pod="opendatahub/model-serving-api-75bd89fdfc-w5msn" Nov 04 05:33:55.687705 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.687520 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cf6a57d-3d4b-4aef-913f-c2fd31147260-metrics-certs\") pod \"odh-model-controller-769875db56-rv57b\" (UID: \"0cf6a57d-3d4b-4aef-913f-c2fd31147260\") " pod="opendatahub/odh-model-controller-769875db56-rv57b" Nov 04 05:33:55.687705 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.687640 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wspsq\" (UniqueName: \"kubernetes.io/projected/8972ea5d-5467-4935-b341-d3fbd7750a21-kube-api-access-wspsq\") pod \"model-serving-api-75bd89fdfc-w5msn\" (UID: \"8972ea5d-5467-4935-b341-d3fbd7750a21\") " pod="opendatahub/model-serving-api-75bd89fdfc-w5msn" Nov 04 05:33:55.687705 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.687658 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cf6a57d-3d4b-4aef-913f-c2fd31147260-cert\") pod \"odh-model-controller-769875db56-rv57b\" (UID: \"0cf6a57d-3d4b-4aef-913f-c2fd31147260\") " pod="opendatahub/odh-model-controller-769875db56-rv57b" Nov 04 05:33:55.687893 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.687726 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sszt5\" (UniqueName: \"kubernetes.io/projected/0cf6a57d-3d4b-4aef-913f-c2fd31147260-kube-api-access-sszt5\") pod \"odh-model-controller-769875db56-rv57b\" (UID: \"0cf6a57d-3d4b-4aef-913f-c2fd31147260\") " pod="opendatahub/odh-model-controller-769875db56-rv57b" Nov 04 05:33:55.690070 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.690041 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8972ea5d-5467-4935-b341-d3fbd7750a21-tls-certs\") pod \"model-serving-api-75bd89fdfc-w5msn\" (UID: \"8972ea5d-5467-4935-b341-d3fbd7750a21\") " pod="opendatahub/model-serving-api-75bd89fdfc-w5msn" Nov 04 05:33:55.706181 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.706144 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-rv57b"] Nov 04 05:33:55.749652 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.749618 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-cggpf\"" Nov 04 05:33:55.749652 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.749637 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Nov 04 05:33:55.749652 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.749628 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\"" Nov 04 05:33:55.753220 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.753195 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wspsq\" (UniqueName: \"kubernetes.io/projected/8972ea5d-5467-4935-b341-d3fbd7750a21-kube-api-access-wspsq\") pod \"model-serving-api-75bd89fdfc-w5msn\" (UID: \"8972ea5d-5467-4935-b341-d3fbd7750a21\") " pod="opendatahub/model-serving-api-75bd89fdfc-w5msn" Nov 04 05:33:55.788382 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.788282 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cf6a57d-3d4b-4aef-913f-c2fd31147260-metrics-certs\") pod \"odh-model-controller-769875db56-rv57b\" (UID: \"0cf6a57d-3d4b-4aef-913f-c2fd31147260\") " pod="opendatahub/odh-model-controller-769875db56-rv57b" Nov 04 05:33:55.788382 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.788339 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cf6a57d-3d4b-4aef-913f-c2fd31147260-cert\") pod \"odh-model-controller-769875db56-rv57b\" (UID: \"0cf6a57d-3d4b-4aef-913f-c2fd31147260\") " pod="opendatahub/odh-model-controller-769875db56-rv57b" Nov 04 05:33:55.788382 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.788361 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sszt5\" (UniqueName: \"kubernetes.io/projected/0cf6a57d-3d4b-4aef-913f-c2fd31147260-kube-api-access-sszt5\") pod \"odh-model-controller-769875db56-rv57b\" (UID: \"0cf6a57d-3d4b-4aef-913f-c2fd31147260\") " pod="opendatahub/odh-model-controller-769875db56-rv57b" Nov 04 05:33:55.791106 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.791078 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cf6a57d-3d4b-4aef-913f-c2fd31147260-cert\") pod \"odh-model-controller-769875db56-rv57b\" (UID: \"0cf6a57d-3d4b-4aef-913f-c2fd31147260\") " pod="opendatahub/odh-model-controller-769875db56-rv57b" Nov 04 05:33:55.791246 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.791197 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cf6a57d-3d4b-4aef-913f-c2fd31147260-metrics-certs\") pod \"odh-model-controller-769875db56-rv57b\" (UID: \"0cf6a57d-3d4b-4aef-913f-c2fd31147260\") " pod="opendatahub/odh-model-controller-769875db56-rv57b" Nov 04 05:33:55.854145 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.854108 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sszt5\" (UniqueName: \"kubernetes.io/projected/0cf6a57d-3d4b-4aef-913f-c2fd31147260-kube-api-access-sszt5\") pod \"odh-model-controller-769875db56-rv57b\" (UID: \"0cf6a57d-3d4b-4aef-913f-c2fd31147260\") " pod="opendatahub/odh-model-controller-769875db56-rv57b" Nov 04 05:33:55.855798 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.855772 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-w5msn" Nov 04 05:33:55.950212 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:55.950017 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-rv57b" Nov 04 05:33:56.252166 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:56.252129 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-w5msn"] Nov 04 05:33:56.256122 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:33:56.256092 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8972ea5d_5467_4935_b341_d3fbd7750a21.slice/crio-e09d72665a572d06bbbab3f03b1e4cb290595523646857c7b4a1a62ad4b519bb WatchSource:0}: Error finding container e09d72665a572d06bbbab3f03b1e4cb290595523646857c7b4a1a62ad4b519bb: Status 404 returned error can't find the container with id e09d72665a572d06bbbab3f03b1e4cb290595523646857c7b4a1a62ad4b519bb Nov 04 05:33:56.376694 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:56.371380 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-rv57b"] Nov 04 05:33:56.380723 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:33:56.380694 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf6a57d_3d4b_4aef_913f_c2fd31147260.slice/crio-5f6f23c74b946aef9b50f2cb931e157a126821e7d10233ac541f9f47e1c5cdc6 WatchSource:0}: Error finding container 5f6f23c74b946aef9b50f2cb931e157a126821e7d10233ac541f9f47e1c5cdc6: Status 404 returned error can't find the container with id 5f6f23c74b946aef9b50f2cb931e157a126821e7d10233ac541f9f47e1c5cdc6 Nov 04 05:33:56.898474 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:56.898431 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-w5msn" event={"ID":"8972ea5d-5467-4935-b341-d3fbd7750a21","Type":"ContainerStarted","Data":"e09d72665a572d06bbbab3f03b1e4cb290595523646857c7b4a1a62ad4b519bb"} Nov 04 05:33:56.899721 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:56.899690 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-rv57b" event={"ID":"0cf6a57d-3d4b-4aef-913f-c2fd31147260","Type":"ContainerStarted","Data":"5f6f23c74b946aef9b50f2cb931e157a126821e7d10233ac541f9f47e1c5cdc6"} Nov 04 05:33:58.361658 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:33:58.361614 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:33:58.913598 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:58.913559 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" event={"ID":"1b132e4f-1cda-4c76-820a-b81feb73ad40","Type":"ContainerStarted","Data":"c7593c8d70ef63f779424972b097f30ec4eabefbdd3115e5877bd9ae4125e9c2"} Nov 04 05:33:58.913799 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:58.913702 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:33:59.040777 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:59.040715 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" podStartSLOduration=2.887459159 podStartE2EDuration="24.040693483s" totalImagesPullingTime="21.153234324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:33:59.040084135 +0000 UTC m=+409.041719861" watchObservedRunningTime="2026-11-04 05:33:59.040693483 +0000 UTC m=+409.042329209" Nov 04 05:33:59.921983 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:59.921944 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-w5msn" event={"ID":"8972ea5d-5467-4935-b341-d3fbd7750a21","Type":"ContainerStarted","Data":"80ba9c1a58263fce8a98f7fd4bb50f137f4b5574e742ac3fe8d433f728189705"} Nov 04 05:33:59.922523 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:33:59.922271 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-w5msn" Nov 04 05:34:00.048948 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:00.048880 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-w5msn" podStartSLOduration=1.7958548589999999 podStartE2EDuration="5.048863367s" totalImagesPullingTime="3.253008508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:33:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:34:00.04714886 +0000 UTC m=+410.048784587" watchObservedRunningTime="2026-11-04 05:34:00.048863367 +0000 UTC m=+410.050499093" Nov 04 05:34:03.942362 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:03.942325 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-rv57b" event={"ID":"0cf6a57d-3d4b-4aef-913f-c2fd31147260","Type":"ContainerStarted","Data":"bf4321089790847d0f11cb3154dd53616fc8a9539a636c460ee5a6ac4995ed28"} Nov 04 05:34:03.942797 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:03.942450 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-rv57b" Nov 04 05:34:04.069435 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:04.069381 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-rv57b" podStartSLOduration=2.578656559 podStartE2EDuration="9.069366859s" totalImagesPullingTime="6.4907103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:33:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:34:04.068638355 +0000 UTC m=+414.070274079" watchObservedRunningTime="2026-11-04 05:34:04.069366859 +0000 UTC m=+414.071002584" Nov 04 05:34:09.043977 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.043943 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl"] Nov 04 05:34:09.047627 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.047606 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:09.110133 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.110104 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/024a4628-c743-447d-8d90-5232a27198f3-maas-api-tls\") pod \"maas-api-674d648f7b-sxqpl\" (UID: \"024a4628-c743-447d-8d90-5232a27198f3\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:09.110133 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.110137 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwwkk\" (UniqueName: \"kubernetes.io/projected/024a4628-c743-447d-8d90-5232a27198f3-kube-api-access-zwwkk\") pod \"maas-api-674d648f7b-sxqpl\" (UID: \"024a4628-c743-447d-8d90-5232a27198f3\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:09.110379 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.110246 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/024a4628-c743-447d-8d90-5232a27198f3-metrics-certs\") pod \"maas-api-674d648f7b-sxqpl\" (UID: \"024a4628-c743-447d-8d90-5232a27198f3\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:09.110379 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.110313 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/024a4628-c743-447d-8d90-5232a27198f3-openshift-service-ca\") pod \"maas-api-674d648f7b-sxqpl\" (UID: \"024a4628-c743-447d-8d90-5232a27198f3\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:09.159548 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.159508 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 05:34:09.159548 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.159548 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Nov 04 05:34:09.159815 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.159559 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8787b\"" Nov 04 05:34:09.159815 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.159747 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Nov 04 05:34:09.159883 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.159821 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 05:34:09.161054 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.161037 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl"] Nov 04 05:34:09.211168 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.211125 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/024a4628-c743-447d-8d90-5232a27198f3-openshift-service-ca\") pod \"maas-api-674d648f7b-sxqpl\" (UID: \"024a4628-c743-447d-8d90-5232a27198f3\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:09.211390 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.211233 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/024a4628-c743-447d-8d90-5232a27198f3-maas-api-tls\") pod \"maas-api-674d648f7b-sxqpl\" (UID: \"024a4628-c743-447d-8d90-5232a27198f3\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:09.211390 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.211264 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwwkk\" (UniqueName: \"kubernetes.io/projected/024a4628-c743-447d-8d90-5232a27198f3-kube-api-access-zwwkk\") pod \"maas-api-674d648f7b-sxqpl\" (UID: \"024a4628-c743-447d-8d90-5232a27198f3\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:09.211632 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.211604 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/024a4628-c743-447d-8d90-5232a27198f3-metrics-certs\") pod \"maas-api-674d648f7b-sxqpl\" (UID: \"024a4628-c743-447d-8d90-5232a27198f3\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:09.212101 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.212050 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/024a4628-c743-447d-8d90-5232a27198f3-openshift-service-ca\") pod \"maas-api-674d648f7b-sxqpl\" (UID: \"024a4628-c743-447d-8d90-5232a27198f3\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:09.214050 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.214026 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/024a4628-c743-447d-8d90-5232a27198f3-maas-api-tls\") pod \"maas-api-674d648f7b-sxqpl\" (UID: \"024a4628-c743-447d-8d90-5232a27198f3\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:09.214428 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.214407 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/024a4628-c743-447d-8d90-5232a27198f3-metrics-certs\") pod \"maas-api-674d648f7b-sxqpl\" (UID: \"024a4628-c743-447d-8d90-5232a27198f3\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:09.276726 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.276691 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwwkk\" (UniqueName: \"kubernetes.io/projected/024a4628-c743-447d-8d90-5232a27198f3-kube-api-access-zwwkk\") pod \"maas-api-674d648f7b-sxqpl\" (UID: \"024a4628-c743-447d-8d90-5232a27198f3\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:09.360425 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.360336 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:09.737551 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.737514 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl"] Nov 04 05:34:09.742400 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:34:09.742374 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod024a4628_c743_447d_8d90_5232a27198f3.slice/crio-f5b28c7e73b88b92fa2ba8b48890cbb0d22815ef626e7218e14a4798f7af893d WatchSource:0}: Error finding container f5b28c7e73b88b92fa2ba8b48890cbb0d22815ef626e7218e14a4798f7af893d: Status 404 returned error can't find the container with id f5b28c7e73b88b92fa2ba8b48890cbb0d22815ef626e7218e14a4798f7af893d Nov 04 05:34:09.928281 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.928247 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cjkv4" Nov 04 05:34:09.970047 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:09.969999 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" event={"ID":"024a4628-c743-447d-8d90-5232a27198f3","Type":"ContainerStarted","Data":"f5b28c7e73b88b92fa2ba8b48890cbb0d22815ef626e7218e14a4798f7af893d"} Nov 04 05:34:10.931224 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:10.931194 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-w5msn" Nov 04 05:34:13.992774 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:13.992687 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" event={"ID":"024a4628-c743-447d-8d90-5232a27198f3","Type":"ContainerStarted","Data":"039225c3acccf9186d3d01519bc663239dd485941b3268236a04e0990eadf952"} Nov 04 05:34:13.993146 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:13.992821 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:14.119089 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:14.119036 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" podStartSLOduration=1.127388276 podStartE2EDuration="5.11902324s" totalImagesPullingTime="3.991634964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:34:14.117907155 +0000 UTC m=+424.119542880" watchObservedRunningTime="2026-11-04 05:34:14.11902324 +0000 UTC m=+424.120658963" Nov 04 05:34:14.950058 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:14.950025 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-rv57b" Nov 04 05:34:20.003609 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:20.003572 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:21.105931 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.105902 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t"] Nov 04 05:34:21.109827 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.109797 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" Nov 04 05:34:21.222323 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.222282 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t"] Nov 04 05:34:21.228739 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.228708 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a691dddb-3a39-4bc1-8add-c5b2668c869d-metrics-certs\") pod \"maas-api-5cf85ff9fd-9mt9t\" (UID: \"a691dddb-3a39-4bc1-8add-c5b2668c869d\") " pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" Nov 04 05:34:21.228739 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.228742 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pcg4\" (UniqueName: \"kubernetes.io/projected/a691dddb-3a39-4bc1-8add-c5b2668c869d-kube-api-access-2pcg4\") pod \"maas-api-5cf85ff9fd-9mt9t\" (UID: \"a691dddb-3a39-4bc1-8add-c5b2668c869d\") " pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" Nov 04 05:34:21.228994 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.228785 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a691dddb-3a39-4bc1-8add-c5b2668c869d-maas-api-tls\") pod \"maas-api-5cf85ff9fd-9mt9t\" (UID: \"a691dddb-3a39-4bc1-8add-c5b2668c869d\") " pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" Nov 04 05:34:21.228994 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.228929 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a691dddb-3a39-4bc1-8add-c5b2668c869d-openshift-service-ca\") pod \"maas-api-5cf85ff9fd-9mt9t\" (UID: \"a691dddb-3a39-4bc1-8add-c5b2668c869d\") " pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" Nov 04 05:34:21.330072 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.330031 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a691dddb-3a39-4bc1-8add-c5b2668c869d-maas-api-tls\") pod \"maas-api-5cf85ff9fd-9mt9t\" (UID: \"a691dddb-3a39-4bc1-8add-c5b2668c869d\") " pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" Nov 04 05:34:21.330292 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.330082 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a691dddb-3a39-4bc1-8add-c5b2668c869d-openshift-service-ca\") pod \"maas-api-5cf85ff9fd-9mt9t\" (UID: \"a691dddb-3a39-4bc1-8add-c5b2668c869d\") " pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" Nov 04 05:34:21.330292 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.330122 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a691dddb-3a39-4bc1-8add-c5b2668c869d-metrics-certs\") pod \"maas-api-5cf85ff9fd-9mt9t\" (UID: \"a691dddb-3a39-4bc1-8add-c5b2668c869d\") " pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" Nov 04 05:34:21.330292 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.330137 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pcg4\" (UniqueName: \"kubernetes.io/projected/a691dddb-3a39-4bc1-8add-c5b2668c869d-kube-api-access-2pcg4\") pod \"maas-api-5cf85ff9fd-9mt9t\" (UID: \"a691dddb-3a39-4bc1-8add-c5b2668c869d\") " pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" Nov 04 05:34:21.330844 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.330816 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a691dddb-3a39-4bc1-8add-c5b2668c869d-openshift-service-ca\") pod \"maas-api-5cf85ff9fd-9mt9t\" (UID: \"a691dddb-3a39-4bc1-8add-c5b2668c869d\") " pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" Nov 04 05:34:21.332575 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.332547 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a691dddb-3a39-4bc1-8add-c5b2668c869d-maas-api-tls\") pod \"maas-api-5cf85ff9fd-9mt9t\" (UID: \"a691dddb-3a39-4bc1-8add-c5b2668c869d\") " pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" Nov 04 05:34:21.332575 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.332573 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a691dddb-3a39-4bc1-8add-c5b2668c869d-metrics-certs\") pod \"maas-api-5cf85ff9fd-9mt9t\" (UID: \"a691dddb-3a39-4bc1-8add-c5b2668c869d\") " pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" Nov 04 05:34:21.395929 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.395852 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pcg4\" (UniqueName: \"kubernetes.io/projected/a691dddb-3a39-4bc1-8add-c5b2668c869d-kube-api-access-2pcg4\") pod \"maas-api-5cf85ff9fd-9mt9t\" (UID: \"a691dddb-3a39-4bc1-8add-c5b2668c869d\") " pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" Nov 04 05:34:21.422561 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.422524 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" Nov 04 05:34:21.799275 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:21.799234 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t"] Nov 04 05:34:21.804175 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:34:21.804142 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda691dddb_3a39_4bc1_8add_c5b2668c869d.slice/crio-baf9e7ebfb3f2c02a47a6aa4924d01e778f27fa0a51cff16b0dfce7353b9dbe7 WatchSource:0}: Error finding container baf9e7ebfb3f2c02a47a6aa4924d01e778f27fa0a51cff16b0dfce7353b9dbe7: Status 404 returned error can't find the container with id baf9e7ebfb3f2c02a47a6aa4924d01e778f27fa0a51cff16b0dfce7353b9dbe7 Nov 04 05:34:22.029399 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:22.029355 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" event={"ID":"a691dddb-3a39-4bc1-8add-c5b2668c869d","Type":"ContainerStarted","Data":"baf9e7ebfb3f2c02a47a6aa4924d01e778f27fa0a51cff16b0dfce7353b9dbe7"} Nov 04 05:34:23.035673 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:23.035622 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" event={"ID":"a691dddb-3a39-4bc1-8add-c5b2668c869d","Type":"ContainerStarted","Data":"6b1ebbb4a9f1e1f682c4e8c2dc322d3810eb24e7da2dfdfaab52a83f58a8d9cd"} Nov 04 05:34:23.036059 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:23.035736 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" Nov 04 05:34:23.167398 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:23.167352 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" podStartSLOduration=1.170169192 podStartE2EDuration="2.167339881s" totalImagesPullingTime="997.170689ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:34:23.162177941 +0000 UTC m=+433.163813666" watchObservedRunningTime="2026-11-04 05:34:23.167339881 +0000 UTC m=+433.168975606" Nov 04 05:34:28.342417 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:34:28.342383 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:34:29.045927 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:29.045898 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5cf85ff9fd-9mt9t" Nov 04 05:34:29.227985 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:29.227943 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl"] Nov 04 05:34:29.228282 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:29.228241 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" podUID="024a4628-c743-447d-8d90-5232a27198f3" containerName="maas-api" containerID="cri-o://039225c3acccf9186d3d01519bc663239dd485941b3268236a04e0990eadf952" gracePeriod=30 Nov 04 05:34:30.012318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.012284 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:30.068551 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.068515 4859 generic.go:356] "Generic (PLEG): container finished" podID="024a4628-c743-447d-8d90-5232a27198f3" containerID="039225c3acccf9186d3d01519bc663239dd485941b3268236a04e0990eadf952" exitCode=0 Nov 04 05:34:30.068769 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.068606 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" Nov 04 05:34:30.068769 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.068610 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" event={"ID":"024a4628-c743-447d-8d90-5232a27198f3","Type":"ContainerDied","Data":"039225c3acccf9186d3d01519bc663239dd485941b3268236a04e0990eadf952"} Nov 04 05:34:30.068769 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.068692 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl" event={"ID":"024a4628-c743-447d-8d90-5232a27198f3","Type":"ContainerDied","Data":"f5b28c7e73b88b92fa2ba8b48890cbb0d22815ef626e7218e14a4798f7af893d"} Nov 04 05:34:30.068769 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.068723 4859 scope.go:122] "RemoveContainer" containerID="039225c3acccf9186d3d01519bc663239dd485941b3268236a04e0990eadf952" Nov 04 05:34:30.077389 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.077372 4859 scope.go:122] "RemoveContainer" containerID="039225c3acccf9186d3d01519bc663239dd485941b3268236a04e0990eadf952" Nov 04 05:34:30.077780 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:34:30.077758 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"039225c3acccf9186d3d01519bc663239dd485941b3268236a04e0990eadf952\": container with ID starting with 039225c3acccf9186d3d01519bc663239dd485941b3268236a04e0990eadf952 not found: ID does not exist" containerID="039225c3acccf9186d3d01519bc663239dd485941b3268236a04e0990eadf952" Nov 04 05:34:30.077831 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.077791 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"039225c3acccf9186d3d01519bc663239dd485941b3268236a04e0990eadf952"} err="failed to get container status \"039225c3acccf9186d3d01519bc663239dd485941b3268236a04e0990eadf952\": rpc error: code = NotFound desc = could not find container \"039225c3acccf9186d3d01519bc663239dd485941b3268236a04e0990eadf952\": container with ID starting with 039225c3acccf9186d3d01519bc663239dd485941b3268236a04e0990eadf952 not found: ID does not exist" Nov 04 05:34:30.107328 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.107242 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/024a4628-c743-447d-8d90-5232a27198f3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/024a4628-c743-447d-8d90-5232a27198f3-openshift-service-ca\") pod \"024a4628-c743-447d-8d90-5232a27198f3\" (UID: \"024a4628-c743-447d-8d90-5232a27198f3\") " Nov 04 05:34:30.107328 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.107290 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/024a4628-c743-447d-8d90-5232a27198f3-kube-api-access-zwwkk\" (UniqueName: \"kubernetes.io/projected/024a4628-c743-447d-8d90-5232a27198f3-kube-api-access-zwwkk\") pod \"024a4628-c743-447d-8d90-5232a27198f3\" (UID: \"024a4628-c743-447d-8d90-5232a27198f3\") " Nov 04 05:34:30.107509 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.107362 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/024a4628-c743-447d-8d90-5232a27198f3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/024a4628-c743-447d-8d90-5232a27198f3-metrics-certs\") pod \"024a4628-c743-447d-8d90-5232a27198f3\" (UID: \"024a4628-c743-447d-8d90-5232a27198f3\") " Nov 04 05:34:30.107509 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.107403 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/024a4628-c743-447d-8d90-5232a27198f3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/024a4628-c743-447d-8d90-5232a27198f3-maas-api-tls\") pod \"024a4628-c743-447d-8d90-5232a27198f3\" (UID: \"024a4628-c743-447d-8d90-5232a27198f3\") " Nov 04 05:34:30.107746 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.107714 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/024a4628-c743-447d-8d90-5232a27198f3-openshift-service-ca" pod "024a4628-c743-447d-8d90-5232a27198f3" (UID: "024a4628-c743-447d-8d90-5232a27198f3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:34:30.109773 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.109743 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/024a4628-c743-447d-8d90-5232a27198f3-maas-api-tls" pod "024a4628-c743-447d-8d90-5232a27198f3" (UID: "024a4628-c743-447d-8d90-5232a27198f3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:34:30.109773 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.109760 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/024a4628-c743-447d-8d90-5232a27198f3-kube-api-access-zwwkk" pod "024a4628-c743-447d-8d90-5232a27198f3" (UID: "024a4628-c743-447d-8d90-5232a27198f3"). InnerVolumeSpecName "kube-api-access-zwwkk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:34:30.109925 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.109832 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/024a4628-c743-447d-8d90-5232a27198f3-metrics-certs" pod "024a4628-c743-447d-8d90-5232a27198f3" (UID: "024a4628-c743-447d-8d90-5232a27198f3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:34:30.209116 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.209074 4859 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/024a4628-c743-447d-8d90-5232a27198f3-metrics-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:34:30.209116 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.209108 4859 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/024a4628-c743-447d-8d90-5232a27198f3-maas-api-tls\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:34:30.209116 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.209120 4859 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/024a4628-c743-447d-8d90-5232a27198f3-openshift-service-ca\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:34:30.209357 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.209129 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwwkk\" (UniqueName: \"kubernetes.io/projected/024a4628-c743-447d-8d90-5232a27198f3-kube-api-access-zwwkk\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:34:30.491413 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.491378 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl"] Nov 04 05:34:30.497276 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:30.497243 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-sxqpl"] Nov 04 05:34:32.114117 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:34:32.114082 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="024a4628-c743-447d-8d90-5232a27198f3" path="/var/lib/kubelet/pods/024a4628-c743-447d-8d90-5232a27198f3/volumes" Nov 04 05:34:58.342003 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:34:58.341963 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:35:28.342078 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:35:28.342040 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:35:58.340989 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:35:58.340954 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:36:02.297799 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.297755 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d"] Nov 04 05:36:02.298626 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.298604 4859 cpu_manager.go:414] "containerMap: removing container" podUID="024a4628-c743-447d-8d90-5232a27198f3" containerName="maas-api" Nov 04 05:36:02.298746 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.298628 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="024a4628-c743-447d-8d90-5232a27198f3" containerName="maas-api" Nov 04 05:36:02.298899 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.298885 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="024a4628-c743-447d-8d90-5232a27198f3" containerName="maas-api" Nov 04 05:36:02.303018 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.302806 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.364525 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.364442 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d"] Nov 04 05:36:02.393786 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.393752 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.393932 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.393794 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.393932 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.393869 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.393932 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.393907 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.394047 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.393983 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.394086 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.394048 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh2rk\" (UniqueName: \"kubernetes.io/projected/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-kube-api-access-nh2rk\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.416407 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.416383 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Nov 04 05:36:02.416520 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.416437 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Nov 04 05:36:02.416520 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.416498 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-528s5\"" Nov 04 05:36:02.416520 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.416514 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Nov 04 05:36:02.495042 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.495011 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.495042 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.495043 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.495292 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.495065 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.495292 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.495087 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.495292 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.495112 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.495292 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.495158 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh2rk\" (UniqueName: \"kubernetes.io/projected/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-kube-api-access-nh2rk\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.495525 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.495503 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.495593 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.495504 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.495593 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.495587 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.497460 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.497434 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.497585 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.497570 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.560934 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.560902 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh2rk\" (UniqueName: \"kubernetes.io/projected/4fdb4e8c-8202-4eb9-80a9-06c460502eb2-kube-api-access-nh2rk\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-6w29d\" (UID: \"4fdb4e8c-8202-4eb9-80a9-06c460502eb2\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.615976 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.615902 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:02.992853 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:02.992817 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d"] Nov 04 05:36:02.997531 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:36:02.997502 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fdb4e8c_8202_4eb9_80a9_06c460502eb2.slice/crio-43fd6285b9f240aaaf08ab97fd13660d207507f3f9f0a4cab639af846011715a WatchSource:0}: Error finding container 43fd6285b9f240aaaf08ab97fd13660d207507f3f9f0a4cab639af846011715a: Status 404 returned error can't find the container with id 43fd6285b9f240aaaf08ab97fd13660d207507f3f9f0a4cab639af846011715a Nov 04 05:36:03.471942 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:03.471907 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" event={"ID":"4fdb4e8c-8202-4eb9-80a9-06c460502eb2","Type":"ContainerStarted","Data":"43fd6285b9f240aaaf08ab97fd13660d207507f3f9f0a4cab639af846011715a"} Nov 04 05:36:05.482522 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:05.482490 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" event={"ID":"4fdb4e8c-8202-4eb9-80a9-06c460502eb2","Type":"ContainerStarted","Data":"359dfebf21c495db4e5fee906ca0cfd528804130c6e96ee55e6483605ae91247"} Nov 04 05:36:05.610071 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:05.610015 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" podStartSLOduration=1.528633575 podStartE2EDuration="3.610000209s" totalImagesPullingTime="2.081366634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:05.609061208 +0000 UTC m=+535.610696934" watchObservedRunningTime="2026-11-04 05:36:05.610000209 +0000 UTC m=+535.611635933" Nov 04 05:36:05.616192 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:05.616162 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:09.164330 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.164296 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w"] Nov 04 05:36:09.167959 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.167942 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.254994 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.254958 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78899087-8b82-4eef-a7b1-70e2adb9665d-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.255169 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.255012 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78899087-8b82-4eef-a7b1-70e2adb9665d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.255169 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.255117 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d5fs\" (UniqueName: \"kubernetes.io/projected/78899087-8b82-4eef-a7b1-70e2adb9665d-kube-api-access-4d5fs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.255169 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.255155 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78899087-8b82-4eef-a7b1-70e2adb9665d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.255297 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.255247 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78899087-8b82-4eef-a7b1-70e2adb9665d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.255297 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.255279 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78899087-8b82-4eef-a7b1-70e2adb9665d-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.279222 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.279190 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Nov 04 05:36:09.280736 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.280713 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w"] Nov 04 05:36:09.356189 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.356148 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78899087-8b82-4eef-a7b1-70e2adb9665d-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.356189 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.356194 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78899087-8b82-4eef-a7b1-70e2adb9665d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.356414 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.356219 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d5fs\" (UniqueName: \"kubernetes.io/projected/78899087-8b82-4eef-a7b1-70e2adb9665d-kube-api-access-4d5fs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.356414 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.356243 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78899087-8b82-4eef-a7b1-70e2adb9665d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.356414 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.356266 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78899087-8b82-4eef-a7b1-70e2adb9665d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.356414 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.356288 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78899087-8b82-4eef-a7b1-70e2adb9665d-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.356608 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.356548 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78899087-8b82-4eef-a7b1-70e2adb9665d-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.356695 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.356632 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78899087-8b82-4eef-a7b1-70e2adb9665d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.356795 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.356774 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78899087-8b82-4eef-a7b1-70e2adb9665d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.358466 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.358448 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78899087-8b82-4eef-a7b1-70e2adb9665d-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.358544 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.358527 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78899087-8b82-4eef-a7b1-70e2adb9665d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.422525 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.422465 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d5fs\" (UniqueName: \"kubernetes.io/projected/78899087-8b82-4eef-a7b1-70e2adb9665d-kube-api-access-4d5fs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w\" (UID: \"78899087-8b82-4eef-a7b1-70e2adb9665d\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.482345 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.482321 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:09.869955 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:09.869878 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w"] Nov 04 05:36:09.873984 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:36:09.873959 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78899087_8b82_4eef_a7b1_70e2adb9665d.slice/crio-4fc4e490c7ec84dfaeae6f3379d5a3be5bdf4e3def463292b43ba958c0115eb1 WatchSource:0}: Error finding container 4fc4e490c7ec84dfaeae6f3379d5a3be5bdf4e3def463292b43ba958c0115eb1: Status 404 returned error can't find the container with id 4fc4e490c7ec84dfaeae6f3379d5a3be5bdf4e3def463292b43ba958c0115eb1 Nov 04 05:36:10.506574 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:10.506539 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" event={"ID":"78899087-8b82-4eef-a7b1-70e2adb9665d","Type":"ContainerStarted","Data":"4fc4e490c7ec84dfaeae6f3379d5a3be5bdf4e3def463292b43ba958c0115eb1"} Nov 04 05:36:11.511913 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:11.511876 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" event={"ID":"78899087-8b82-4eef-a7b1-70e2adb9665d","Type":"ContainerStarted","Data":"a57b1bb447952f95bfffb21d110c2c340db8bdc17b4bf6360bfce6a52433686a"} Nov 04 05:36:11.645220 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:11.645169 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" podStartSLOduration=2.031338872 podStartE2EDuration="2.645154251s" totalImagesPullingTime="613.815379ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:11.643592306 +0000 UTC m=+541.645228032" watchObservedRunningTime="2026-11-04 05:36:11.645154251 +0000 UTC m=+541.646789975" Nov 04 05:36:12.483330 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:12.483293 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:12.617111 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:12.617079 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:12.629499 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:12.629474 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:13.531100 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:13.531072 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-6w29d" Nov 04 05:36:16.399931 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.399883 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8"] Nov 04 05:36:16.403680 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.403644 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.516842 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.516811 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e15551cb-d922-4daf-b031-bfe47a443e54-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.516989 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.516849 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e15551cb-d922-4daf-b031-bfe47a443e54-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.516989 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.516888 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e15551cb-d922-4daf-b031-bfe47a443e54-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.516989 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.516931 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7fh7\" (UniqueName: \"kubernetes.io/projected/e15551cb-d922-4daf-b031-bfe47a443e54-kube-api-access-n7fh7\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.517133 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.517070 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e15551cb-d922-4daf-b031-bfe47a443e54-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.517133 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.517098 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e15551cb-d922-4daf-b031-bfe47a443e54-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.517307 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.517289 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Nov 04 05:36:16.518924 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.518906 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8"] Nov 04 05:36:16.617879 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.617840 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e15551cb-d922-4daf-b031-bfe47a443e54-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.617879 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.617878 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e15551cb-d922-4daf-b031-bfe47a443e54-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.618071 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.617906 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e15551cb-d922-4daf-b031-bfe47a443e54-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.618071 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.617922 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7fh7\" (UniqueName: \"kubernetes.io/projected/e15551cb-d922-4daf-b031-bfe47a443e54-kube-api-access-n7fh7\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.618071 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.617963 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e15551cb-d922-4daf-b031-bfe47a443e54-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.618071 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.617979 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e15551cb-d922-4daf-b031-bfe47a443e54-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.618339 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.618311 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e15551cb-d922-4daf-b031-bfe47a443e54-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.618451 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.618347 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e15551cb-d922-4daf-b031-bfe47a443e54-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.618451 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.618364 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e15551cb-d922-4daf-b031-bfe47a443e54-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.620220 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.620200 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e15551cb-d922-4daf-b031-bfe47a443e54-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.620516 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.620496 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e15551cb-d922-4daf-b031-bfe47a443e54-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.685100 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.685074 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7fh7\" (UniqueName: \"kubernetes.io/projected/e15551cb-d922-4daf-b031-bfe47a443e54-kube-api-access-n7fh7\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-k6jv8\" (UID: \"e15551cb-d922-4daf-b031-bfe47a443e54\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:16.715914 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:16.715887 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:17.102197 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:17.102122 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8"] Nov 04 05:36:17.105118 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:36:17.105091 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode15551cb_d922_4daf_b031_bfe47a443e54.slice/crio-40bdf7061c08ee6b7362fc6c792d2344351df80868d8368c8a4f78340612e13e WatchSource:0}: Error finding container 40bdf7061c08ee6b7362fc6c792d2344351df80868d8368c8a4f78340612e13e: Status 404 returned error can't find the container with id 40bdf7061c08ee6b7362fc6c792d2344351df80868d8368c8a4f78340612e13e Nov 04 05:36:17.539061 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:17.539031 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" event={"ID":"e15551cb-d922-4daf-b031-bfe47a443e54","Type":"ContainerStarted","Data":"40bdf7061c08ee6b7362fc6c792d2344351df80868d8368c8a4f78340612e13e"} Nov 04 05:36:18.545290 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:18.545258 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" event={"ID":"e15551cb-d922-4daf-b031-bfe47a443e54","Type":"ContainerStarted","Data":"1519c0de2b5ec380615cff4257e8efc644d29587c0d37d978005fddeb2dc89e9"} Nov 04 05:36:18.675203 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:18.675147 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" podStartSLOduration=2.066506464 podStartE2EDuration="2.675133541s" totalImagesPullingTime="608.627077ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:18.674274386 +0000 UTC m=+548.675910108" watchObservedRunningTime="2026-11-04 05:36:18.675133541 +0000 UTC m=+548.676769266" Nov 04 05:36:18.716003 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:18.715970 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:19.483374 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:19.483343 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:19.496031 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:19.496007 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:19.561379 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:19.561343 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w" Nov 04 05:36:21.446516 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.446486 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k"] Nov 04 05:36:21.450065 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.450049 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.562086 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.562057 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.562235 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.562101 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.562280 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.562223 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.562280 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.562249 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.562280 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.562267 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.562371 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.562318 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5ppv\" (UniqueName: \"kubernetes.io/projected/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-kube-api-access-x5ppv\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.563264 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.563251 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Nov 04 05:36:21.565060 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.565040 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k"] Nov 04 05:36:21.663100 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.663068 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.663100 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.663100 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.663289 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.663117 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.663289 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.663140 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5ppv\" (UniqueName: \"kubernetes.io/projected/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-kube-api-access-x5ppv\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.663289 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.663258 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.663442 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.663316 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.663546 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.663524 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.663610 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.663541 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.663699 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.663613 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.665558 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.665537 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.665792 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.665773 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.730528 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.730451 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5ppv\" (UniqueName: \"kubernetes.io/projected/8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3-kube-api-access-x5ppv\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t6p2k\" (UID: \"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:21.761965 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:21.761936 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:22.151314 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:22.151242 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k"] Nov 04 05:36:22.155148 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:36:22.155117 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b7836b7_c6dd_413a_bdc5_f5e98e9dcac3.slice/crio-16c0fdba857e03f8f2d7fe60ce976608a940b7dee67a83d931eef697f3ee8cc2 WatchSource:0}: Error finding container 16c0fdba857e03f8f2d7fe60ce976608a940b7dee67a83d931eef697f3ee8cc2: Status 404 returned error can't find the container with id 16c0fdba857e03f8f2d7fe60ce976608a940b7dee67a83d931eef697f3ee8cc2 Nov 04 05:36:22.565641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:22.565606 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" event={"ID":"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3","Type":"ContainerStarted","Data":"16c0fdba857e03f8f2d7fe60ce976608a940b7dee67a83d931eef697f3ee8cc2"} Nov 04 05:36:23.571597 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:23.571560 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" event={"ID":"8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3","Type":"ContainerStarted","Data":"2a99866974cc1fcfc76cceeab97a0e088a3ec1558c535297d2fdccc725072b23"} Nov 04 05:36:23.701681 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:23.701609 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" podStartSLOduration=2.119589224 podStartE2EDuration="2.701594772s" totalImagesPullingTime="582.005548ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:23.700061639 +0000 UTC m=+553.701697365" watchObservedRunningTime="2026-11-04 05:36:23.701594772 +0000 UTC m=+553.703230496" Nov 04 05:36:23.762609 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:23.762579 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:26.716862 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:26.716820 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:26.729328 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:26.729298 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:27.601850 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:27.601821 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-k6jv8" Nov 04 05:36:28.340453 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:36:28.340414 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:36:30.855311 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:30.855276 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-ggvl5"] Nov 04 05:36:30.858698 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:30.858682 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:30.926010 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:30.925983 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\"" Nov 04 05:36:30.952213 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:30.952171 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4282bcb-4054-4202-8513-1392d158ae81-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:30.952368 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:30.952229 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4282bcb-4054-4202-8513-1392d158ae81-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:30.952412 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:30.952370 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4282bcb-4054-4202-8513-1392d158ae81-home\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:30.952454 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:30.952424 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4282bcb-4054-4202-8513-1392d158ae81-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:30.952504 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:30.952492 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4282bcb-4054-4202-8513-1392d158ae81-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:30.952542 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:30.952515 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dblpb\" (UniqueName: \"kubernetes.io/projected/e4282bcb-4054-4202-8513-1392d158ae81-kube-api-access-dblpb\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:30.981811 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:30.981786 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-ggvl5"] Nov 04 05:36:31.053016 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.052984 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4282bcb-4054-4202-8513-1392d158ae81-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:31.053016 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.053023 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dblpb\" (UniqueName: \"kubernetes.io/projected/e4282bcb-4054-4202-8513-1392d158ae81-kube-api-access-dblpb\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:31.053240 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.053068 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4282bcb-4054-4202-8513-1392d158ae81-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:31.053240 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.053106 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4282bcb-4054-4202-8513-1392d158ae81-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:31.053240 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.053124 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4282bcb-4054-4202-8513-1392d158ae81-home\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:31.053240 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.053146 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4282bcb-4054-4202-8513-1392d158ae81-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:31.053570 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.053538 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4282bcb-4054-4202-8513-1392d158ae81-home\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:31.053742 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.053594 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4282bcb-4054-4202-8513-1392d158ae81-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:31.053742 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.053646 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4282bcb-4054-4202-8513-1392d158ae81-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:31.055233 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.055217 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4282bcb-4054-4202-8513-1392d158ae81-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:31.055533 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.055515 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4282bcb-4054-4202-8513-1392d158ae81-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:31.119269 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.119195 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dblpb\" (UniqueName: \"kubernetes.io/projected/e4282bcb-4054-4202-8513-1392d158ae81-kube-api-access-dblpb\") pod \"echo-simulator-kserve-64b6f8f96c-ggvl5\" (UID: \"e4282bcb-4054-4202-8513-1392d158ae81\") " pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:31.171576 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.171543 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:31.561089 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.561037 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-ggvl5"] Nov 04 05:36:31.564822 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:36:31.564795 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4282bcb_4054_4202_8513_1392d158ae81.slice/crio-487656146bbb945d2a3024ecb9f94d503f44e4f899be4ba1983e694b78ada6bb WatchSource:0}: Error finding container 487656146bbb945d2a3024ecb9f94d503f44e4f899be4ba1983e694b78ada6bb: Status 404 returned error can't find the container with id 487656146bbb945d2a3024ecb9f94d503f44e4f899be4ba1983e694b78ada6bb Nov 04 05:36:31.615260 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.615231 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" event={"ID":"e4282bcb-4054-4202-8513-1392d158ae81","Type":"ContainerStarted","Data":"487656146bbb945d2a3024ecb9f94d503f44e4f899be4ba1983e694b78ada6bb"} Nov 04 05:36:31.762866 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.762822 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:31.775262 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:31.775239 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:32.632608 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:32.632581 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t6p2k" Nov 04 05:36:32.840429 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:32.840392 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq"] Nov 04 05:36:32.844252 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:32.844229 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:32.944880 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:32.944830 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq"] Nov 04 05:36:32.973549 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:32.973517 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:32.973749 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:32.973584 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:32.973749 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:32.973638 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:32.973749 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:32.973700 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:32.973749 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:32.973725 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57a4e230-bf07-4f13-9175-b3589b86ecca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:32.973982 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:32.973826 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:32.973982 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:32.973889 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l597f\" (UniqueName: \"kubernetes.io/projected/57a4e230-bf07-4f13-9175-b3589b86ecca-kube-api-access-l597f\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:32.974085 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:32.974046 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:32.974143 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:32.974086 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:32.981708 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:32.981659 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-ztc4z\"" Nov 04 05:36:33.075421 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.075385 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.075421 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.075427 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.075676 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.075484 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.075676 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.075524 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.075676 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.075547 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57a4e230-bf07-4f13-9175-b3589b86ecca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.075676 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.075623 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.075676 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.075652 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l597f\" (UniqueName: \"kubernetes.io/projected/57a4e230-bf07-4f13-9175-b3589b86ecca-kube-api-access-l597f\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.075934 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.075712 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.075934 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.075748 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.075934 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.075884 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.076075 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.076037 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.076268 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.076237 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.076268 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.076259 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.076643 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.076607 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57a4e230-bf07-4f13-9175-b3589b86ecca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.078998 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.078976 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.079213 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.079191 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.142168 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.142137 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.142557 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.142535 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l597f\" (UniqueName: \"kubernetes.io/projected/57a4e230-bf07-4f13-9175-b3589b86ecca-kube-api-access-l597f\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5s5sq\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.161862 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.161832 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:33.618571 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:33.618487 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq"] Nov 04 05:36:33.621989 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:36:33.621960 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57a4e230_bf07_4f13_9175_b3589b86ecca.slice/crio-1fc23f91387d37d3cb1f1bb93859045c30173e303707c59d0ff18d4981d345d6 WatchSource:0}: Error finding container 1fc23f91387d37d3cb1f1bb93859045c30173e303707c59d0ff18d4981d345d6: Status 404 returned error can't find the container with id 1fc23f91387d37d3cb1f1bb93859045c30173e303707c59d0ff18d4981d345d6 Nov 04 05:36:34.633311 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:34.633281 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" event={"ID":"57a4e230-bf07-4f13-9175-b3589b86ecca","Type":"ContainerStarted","Data":"1fc23f91387d37d3cb1f1bb93859045c30173e303707c59d0ff18d4981d345d6"} Nov 04 05:36:34.634896 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:34.634870 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" event={"ID":"e4282bcb-4054-4202-8513-1392d158ae81","Type":"ContainerStarted","Data":"c72b7e54f61c50d17d8e2bfcfa2011efa329d88a77847e0b3f4bccead765e567"} Nov 04 05:36:34.763075 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:34.763023 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" podStartSLOduration=2.206271712 podStartE2EDuration="4.763007802s" totalImagesPullingTime="2.55673609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:34.760962891 +0000 UTC m=+564.762598615" watchObservedRunningTime="2026-11-04 05:36:34.763007802 +0000 UTC m=+564.764643526" Nov 04 05:36:35.172488 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:35.172448 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:38.092087 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:38.092059 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:36:38.092358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:38.092124 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:36:38.092358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:38.092152 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:36:38.655774 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:38.655736 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" event={"ID":"57a4e230-bf07-4f13-9175-b3589b86ecca","Type":"ContainerStarted","Data":"af7ac12f2352f996525146febbbf4ac47620024f73789590d4010ce03c1ba356"} Nov 04 05:36:38.902434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:38.902371 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" podStartSLOduration=2.43525958 podStartE2EDuration="6.902355034s" totalImagesPullingTime="4.467095454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:38.838003194 +0000 UTC m=+568.839638918" watchObservedRunningTime="2026-11-04 05:36:38.902355034 +0000 UTC m=+568.903990760" Nov 04 05:36:39.162345 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:39.162308 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:39.163920 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:39.163895 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5s5sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:36:39.164031 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:39.163948 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" podUID="57a4e230-bf07-4f13-9175-b3589b86ecca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Nov 04 05:36:40.163224 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:40.163193 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5s5sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:36:40.163635 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:40.163253 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" podUID="57a4e230-bf07-4f13-9175-b3589b86ecca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Nov 04 05:36:41.163210 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:41.163174 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5s5sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:36:41.163364 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:41.163234 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" podUID="57a4e230-bf07-4f13-9175-b3589b86ecca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Nov 04 05:36:41.172410 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:41.172387 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:41.201930 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:41.201903 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:41.703104 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:41.703069 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-ggvl5" Nov 04 05:36:42.163304 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:42.163275 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5s5sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:36:42.163457 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:42.163348 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" podUID="57a4e230-bf07-4f13-9175-b3589b86ecca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Nov 04 05:36:43.162961 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:43.162925 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5s5sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:36:43.163134 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:43.162983 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" podUID="57a4e230-bf07-4f13-9175-b3589b86ecca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Nov 04 05:36:44.163053 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:44.163017 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5s5sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:36:44.163435 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:44.163081 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" podUID="57a4e230-bf07-4f13-9175-b3589b86ecca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Nov 04 05:36:45.162866 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:45.162828 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5s5sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:36:45.163040 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:45.162933 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" podUID="57a4e230-bf07-4f13-9175-b3589b86ecca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Nov 04 05:36:46.593053 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:46.593017 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:46.593522 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:46.593368 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:46.594328 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:46.594305 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:36:47.944314 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:47.944282 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524"] Nov 04 05:36:47.949575 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:47.949556 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.007301 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.007267 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.007437 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.007318 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.007437 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.007397 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.007559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.007517 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d8417ad-b883-4d9e-9b80-d352d45202c4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.007608 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.007558 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.007656 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.007619 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.007656 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.007637 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99lbh\" (UniqueName: \"kubernetes.io/projected/1d8417ad-b883-4d9e-9b80-d352d45202c4-kube-api-access-99lbh\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.007774 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.007748 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.007810 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.007798 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.062771 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.062734 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524"] Nov 04 05:36:48.108795 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.108765 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d8417ad-b883-4d9e-9b80-d352d45202c4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.108959 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.108806 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.109009 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.108973 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.109059 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.109006 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99lbh\" (UniqueName: \"kubernetes.io/projected/1d8417ad-b883-4d9e-9b80-d352d45202c4-kube-api-access-99lbh\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.109121 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.109059 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.109121 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.109093 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.109227 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.109153 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.109227 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.109207 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.109334 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.109253 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.109454 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.109431 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.109520 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.109451 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.109642 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.109619 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.109734 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.109640 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d8417ad-b883-4d9e-9b80-d352d45202c4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.109734 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.109722 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.111406 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.111387 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.111502 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.111476 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.174918 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.174893 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.175076 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.175014 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99lbh\" (UniqueName: \"kubernetes.io/projected/1d8417ad-b883-4d9e-9b80-d352d45202c4-kube-api-access-99lbh\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s2524\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.262681 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.262594 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:48.708396 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.708362 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524"] Nov 04 05:36:48.712315 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:36:48.712291 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d8417ad_b883_4d9e_9b80_d352d45202c4.slice/crio-0e5c2d1934dd0ccd837210d95aebc49a99804eaf86c4c494f9f858dec8ec5c29 WatchSource:0}: Error finding container 0e5c2d1934dd0ccd837210d95aebc49a99804eaf86c4c494f9f858dec8ec5c29: Status 404 returned error can't find the container with id 0e5c2d1934dd0ccd837210d95aebc49a99804eaf86c4c494f9f858dec8ec5c29 Nov 04 05:36:48.714522 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.714491 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:36:48.714587 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.714557 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:36:48.714628 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.714589 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:36:48.733345 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:48.733321 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" event={"ID":"1d8417ad-b883-4d9e-9b80-d352d45202c4","Type":"ContainerStarted","Data":"0e5c2d1934dd0ccd837210d95aebc49a99804eaf86c4c494f9f858dec8ec5c29"} Nov 04 05:36:49.740096 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:49.740064 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" event={"ID":"1d8417ad-b883-4d9e-9b80-d352d45202c4","Type":"ContainerStarted","Data":"180db0947f31576d3bbf23eab43a9ad3492fb8d6c0b9d9213905249128f90391"} Nov 04 05:36:49.994767 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:49.985093 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" podStartSLOduration=2.9850747330000003 podStartE2EDuration="2.985074733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:49.92780037 +0000 UTC m=+579.929436095" watchObservedRunningTime="2026-11-04 05:36:49.985074733 +0000 UTC m=+579.986710461" Nov 04 05:36:50.262933 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:50.262827 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:50.264323 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:50.264295 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-s2524 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:36:50.264434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:50.264339 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" podUID="1d8417ad-b883-4d9e-9b80-d352d45202c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Nov 04 05:36:51.263563 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:51.263528 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-s2524 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:36:51.263998 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:51.263601 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" podUID="1d8417ad-b883-4d9e-9b80-d352d45202c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Nov 04 05:36:52.262957 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:52.262924 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-s2524 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:36:52.263127 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:52.262975 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" podUID="1d8417ad-b883-4d9e-9b80-d352d45202c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Nov 04 05:36:53.263253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:53.263215 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-s2524 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:36:53.263628 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:53.263272 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" podUID="1d8417ad-b883-4d9e-9b80-d352d45202c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Nov 04 05:36:54.263469 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:54.263433 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-s2524 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:36:54.263896 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:54.263494 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" podUID="1d8417ad-b883-4d9e-9b80-d352d45202c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Nov 04 05:36:55.263207 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:55.263172 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-s2524 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:36:55.263380 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:55.263253 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" podUID="1d8417ad-b883-4d9e-9b80-d352d45202c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Nov 04 05:36:56.263452 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:56.263413 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-s2524 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:36:56.263837 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:56.263480 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" podUID="1d8417ad-b883-4d9e-9b80-d352d45202c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Nov 04 05:36:57.268749 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:57.268719 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:57.269229 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:57.269064 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:57.269885 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:36:57.269864 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:36:58.340730 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:36:58.340693 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:37:04.748372 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:04.748311 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-79567948bc-gln67"] Nov 04 05:37:04.754044 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:04.754026 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-79567948bc-gln67" Nov 04 05:37:04.811035 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:04.811002 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\"" Nov 04 05:37:04.811035 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:04.811007 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\"" Nov 04 05:37:04.865254 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:04.865220 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-gzmwp\"" Nov 04 05:37:04.866885 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:04.866863 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-79567948bc-gln67"] Nov 04 05:37:04.874676 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:04.874642 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4p4v\" (UniqueName: \"kubernetes.io/projected/7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8-kube-api-access-l4p4v\") pod \"ai-gateway-controller-79567948bc-gln67\" (UID: \"7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8\") " pod="opendatahub/ai-gateway-controller-79567948bc-gln67" Nov 04 05:37:04.874798 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:04.874783 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8-metrics-certs\") pod \"ai-gateway-controller-79567948bc-gln67\" (UID: \"7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8\") " pod="opendatahub/ai-gateway-controller-79567948bc-gln67" Nov 04 05:37:04.975493 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:04.975460 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8-metrics-certs\") pod \"ai-gateway-controller-79567948bc-gln67\" (UID: \"7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8\") " pod="opendatahub/ai-gateway-controller-79567948bc-gln67" Nov 04 05:37:04.975656 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:04.975555 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4p4v\" (UniqueName: \"kubernetes.io/projected/7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8-kube-api-access-l4p4v\") pod \"ai-gateway-controller-79567948bc-gln67\" (UID: \"7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8\") " pod="opendatahub/ai-gateway-controller-79567948bc-gln67" Nov 04 05:37:04.977908 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:04.977889 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8-metrics-certs\") pod \"ai-gateway-controller-79567948bc-gln67\" (UID: \"7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8\") " pod="opendatahub/ai-gateway-controller-79567948bc-gln67" Nov 04 05:37:05.041416 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:05.041345 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4p4v\" (UniqueName: \"kubernetes.io/projected/7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8-kube-api-access-l4p4v\") pod \"ai-gateway-controller-79567948bc-gln67\" (UID: \"7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8\") " pod="opendatahub/ai-gateway-controller-79567948bc-gln67" Nov 04 05:37:05.066123 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:05.066098 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-79567948bc-gln67" Nov 04 05:37:05.451991 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:05.451959 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-79567948bc-gln67"] Nov 04 05:37:05.456021 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:37:05.455995 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f8e0fec_b7d5_43d7_b893_8b9f21beb7f8.slice/crio-ab1996d2fb2abdd75acb21dc792e3a709b2d4f726465bfcac5391d20c765787a WatchSource:0}: Error finding container ab1996d2fb2abdd75acb21dc792e3a709b2d4f726465bfcac5391d20c765787a: Status 404 returned error can't find the container with id ab1996d2fb2abdd75acb21dc792e3a709b2d4f726465bfcac5391d20c765787a Nov 04 05:37:05.819170 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:05.819080 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-79567948bc-gln67" event={"ID":"7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8","Type":"ContainerStarted","Data":"ab1996d2fb2abdd75acb21dc792e3a709b2d4f726465bfcac5391d20c765787a"} Nov 04 05:37:10.157961 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:10.157932 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/2.log" Nov 04 05:37:10.159337 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:10.159313 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/2.log" Nov 04 05:37:11.886134 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:11.886101 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\"" Nov 04 05:37:12.851654 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:12.851614 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-79567948bc-gln67" event={"ID":"7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8","Type":"ContainerStarted","Data":"e65248c3e7a4d1fd55ba403c32e8e34db450de01802b07e0ed36c3363797aa86"} Nov 04 05:37:12.852024 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:12.851705 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-79567948bc-gln67" Nov 04 05:37:12.977018 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:12.976961 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-79567948bc-gln67" podStartSLOduration=2.604598968 podStartE2EDuration="8.976947828s" totalImagesPullingTime="6.37234886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:37:12.975903802 +0000 UTC m=+602.977539529" watchObservedRunningTime="2026-11-04 05:37:12.976947828 +0000 UTC m=+602.978583553" Nov 04 05:37:15.291631 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:15.291594 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-chjgw"] Nov 04 05:37:15.295410 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:15.295394 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-chjgw" Nov 04 05:37:15.351568 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:15.351540 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Nov 04 05:37:15.351948 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:15.351926 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9xr6h\"" Nov 04 05:37:15.361218 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:15.361192 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-chjgw"] Nov 04 05:37:15.476645 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:15.476608 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78647993-f015-4d14-a8df-d6f640bc9508-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-chjgw\" (UID: \"78647993-f015-4d14-a8df-d6f640bc9508\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-chjgw" Nov 04 05:37:15.476876 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:15.476695 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75ds7\" (UniqueName: \"kubernetes.io/projected/78647993-f015-4d14-a8df-d6f640bc9508-kube-api-access-75ds7\") pod \"payload-pre-processing-7b98bb678c-chjgw\" (UID: \"78647993-f015-4d14-a8df-d6f640bc9508\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-chjgw" Nov 04 05:37:15.578421 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:15.578343 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78647993-f015-4d14-a8df-d6f640bc9508-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-chjgw\" (UID: \"78647993-f015-4d14-a8df-d6f640bc9508\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-chjgw" Nov 04 05:37:15.578568 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:15.578424 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75ds7\" (UniqueName: \"kubernetes.io/projected/78647993-f015-4d14-a8df-d6f640bc9508-kube-api-access-75ds7\") pod \"payload-pre-processing-7b98bb678c-chjgw\" (UID: \"78647993-f015-4d14-a8df-d6f640bc9508\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-chjgw" Nov 04 05:37:15.579046 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:15.579026 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78647993-f015-4d14-a8df-d6f640bc9508-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-chjgw\" (UID: \"78647993-f015-4d14-a8df-d6f640bc9508\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-chjgw" Nov 04 05:37:15.643863 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:15.643833 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75ds7\" (UniqueName: \"kubernetes.io/projected/78647993-f015-4d14-a8df-d6f640bc9508-kube-api-access-75ds7\") pod \"payload-pre-processing-7b98bb678c-chjgw\" (UID: \"78647993-f015-4d14-a8df-d6f640bc9508\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-chjgw" Nov 04 05:37:15.908034 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:15.907940 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-chjgw" Nov 04 05:37:16.291939 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:16.291896 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-chjgw"] Nov 04 05:37:16.297359 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:37:16.297333 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78647993_f015_4d14_a8df_d6f640bc9508.slice/crio-bf53845762ce0571ee584451e6e4c5809636c488370e15f714f2d24a48f2288d WatchSource:0}: Error finding container bf53845762ce0571ee584451e6e4c5809636c488370e15f714f2d24a48f2288d: Status 404 returned error can't find the container with id bf53845762ce0571ee584451e6e4c5809636c488370e15f714f2d24a48f2288d Nov 04 05:37:16.873272 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:16.873231 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-chjgw" event={"ID":"78647993-f015-4d14-a8df-d6f640bc9508","Type":"ContainerStarted","Data":"bf53845762ce0571ee584451e6e4c5809636c488370e15f714f2d24a48f2288d"} Nov 04 05:37:17.650826 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:17.650783 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5s5sq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 05:37:17.651243 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:17.650844 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" podUID="57a4e230-bf07-4f13-9175-b3589b86ecca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 05:37:19.889187 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:19.889155 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-chjgw" event={"ID":"78647993-f015-4d14-a8df-d6f640bc9508","Type":"ContainerStarted","Data":"327770ed390733d152aa85e7eabbc64fd78a7ff86ee8a55c3130a7893fe3963b"} Nov 04 05:37:19.889595 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:19.889257 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-chjgw" Nov 04 05:37:20.014638 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:20.014536 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-chjgw" podStartSLOduration=1.6447894760000001 podStartE2EDuration="5.014522213s" totalImagesPullingTime="3.369732737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:37:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:37:20.013440913 +0000 UTC m=+610.015076636" watchObservedRunningTime="2026-11-04 05:37:20.014522213 +0000 UTC m=+610.016157937" Nov 04 05:37:23.859024 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:23.858997 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-79567948bc-gln67" Nov 04 05:37:28.340996 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:37:28.340957 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:37:30.902739 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:30.902705 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-chjgw" Nov 04 05:37:33.345317 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.345279 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc"] Nov 04 05:37:33.349186 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.349163 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.399175 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.399147 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz"] Nov 04 05:37:33.402515 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.402500 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.469838 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.469806 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.470003 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.469843 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be1382e4-d585-4e8a-b5e5-f072ea92f29a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.470003 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.469864 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.470003 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.469935 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.470127 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.470070 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be1382e4-d585-4e8a-b5e5-f072ea92f29a-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.470127 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.470098 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.470211 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.470141 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgrtj\" (UniqueName: \"kubernetes.io/projected/be1382e4-d585-4e8a-b5e5-f072ea92f29a-kube-api-access-qgrtj\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.470211 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.470186 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.470291 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.470247 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be1382e4-d585-4e8a-b5e5-f072ea92f29a-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.470291 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.470264 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.470389 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.470287 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pkqq\" (UniqueName: \"kubernetes.io/projected/52754caf-6be9-4c0f-a8f4-52c2aedf159e-kube-api-access-2pkqq\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.470389 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.470314 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be1382e4-d585-4e8a-b5e5-f072ea92f29a-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.470460 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.470424 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be1382e4-d585-4e8a-b5e5-f072ea92f29a-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.470499 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.470461 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.470499 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.470477 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be1382e4-d585-4e8a-b5e5-f072ea92f29a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.470499 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.470495 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be1382e4-d585-4e8a-b5e5-f072ea92f29a-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.470606 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.470519 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be1382e4-d585-4e8a-b5e5-f072ea92f29a-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.470606 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.470537 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.569241 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.569211 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc"] Nov 04 05:37:33.571207 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571184 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.571343 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571210 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be1382e4-d585-4e8a-b5e5-f072ea92f29a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.571343 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571234 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.571343 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571254 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.571343 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571275 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be1382e4-d585-4e8a-b5e5-f072ea92f29a-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.571343 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571310 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.571580 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571349 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgrtj\" (UniqueName: \"kubernetes.io/projected/be1382e4-d585-4e8a-b5e5-f072ea92f29a-kube-api-access-qgrtj\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.571580 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571472 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.571580 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571534 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be1382e4-d585-4e8a-b5e5-f072ea92f29a-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.571580 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571559 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.571784 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571597 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pkqq\" (UniqueName: \"kubernetes.io/projected/52754caf-6be9-4c0f-a8f4-52c2aedf159e-kube-api-access-2pkqq\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.571784 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571643 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be1382e4-d585-4e8a-b5e5-f072ea92f29a-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.571784 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571657 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.571784 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571705 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be1382e4-d585-4e8a-b5e5-f072ea92f29a-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.571784 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571757 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.571987 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571784 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be1382e4-d585-4e8a-b5e5-f072ea92f29a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.571987 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571813 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be1382e4-d585-4e8a-b5e5-f072ea92f29a-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.571987 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571857 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be1382e4-d585-4e8a-b5e5-f072ea92f29a-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.571987 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571883 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.572175 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.571971 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be1382e4-d585-4e8a-b5e5-f072ea92f29a-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.572175 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.572123 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be1382e4-d585-4e8a-b5e5-f072ea92f29a-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.572775 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.572699 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be1382e4-d585-4e8a-b5e5-f072ea92f29a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.572881 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.572801 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be1382e4-d585-4e8a-b5e5-f072ea92f29a-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.572957 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.572879 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be1382e4-d585-4e8a-b5e5-f072ea92f29a-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.573285 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.573239 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.573409 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.573379 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.573787 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.573766 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.573979 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.573960 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.574774 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.574750 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be1382e4-d585-4e8a-b5e5-f072ea92f29a-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.574857 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.574779 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.574959 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.574941 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be1382e4-d585-4e8a-b5e5-f072ea92f29a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.575232 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.575211 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.637490 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.637415 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.651280 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.651249 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be1382e4-d585-4e8a-b5e5-f072ea92f29a-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.651437 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.651349 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgrtj\" (UniqueName: \"kubernetes.io/projected/be1382e4-d585-4e8a-b5e5-f072ea92f29a-kube-api-access-qgrtj\") pod \"maas-default-gateway-openshift-default-778b7b5c47-t77lc\" (UID: \"be1382e4-d585-4e8a-b5e5-f072ea92f29a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.651505 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.651467 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pkqq\" (UniqueName: \"kubernetes.io/projected/52754caf-6be9-4c0f-a8f4-52c2aedf159e-kube-api-access-2pkqq\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jrfvz\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:33.662255 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.662231 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:33.710568 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.710528 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz"] Nov 04 05:37:33.715707 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:33.715477 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:34.114888 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:34.114856 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc"] Nov 04 05:37:34.117210 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:34.117185 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz"] Nov 04 05:37:34.118607 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:37:34.118556 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe1382e4_d585_4e8a_b5e5_f072ea92f29a.slice/crio-10b580275d36089dd8befa6a0b1f33d8537956aec265ffba565a0a5cafe6cbe3 WatchSource:0}: Error finding container 10b580275d36089dd8befa6a0b1f33d8537956aec265ffba565a0a5cafe6cbe3: Status 404 returned error can't find the container with id 10b580275d36089dd8befa6a0b1f33d8537956aec265ffba565a0a5cafe6cbe3 Nov 04 05:37:34.120520 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:37:34.120499 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52754caf_6be9_4c0f_a8f4_52c2aedf159e.slice/crio-1055e520f1780cd163fb5bd3dcfc3c91de37be21a4033aa3faa1a5fc1bc2f997 WatchSource:0}: Error finding container 1055e520f1780cd163fb5bd3dcfc3c91de37be21a4033aa3faa1a5fc1bc2f997: Status 404 returned error can't find the container with id 1055e520f1780cd163fb5bd3dcfc3c91de37be21a4033aa3faa1a5fc1bc2f997 Nov 04 05:37:34.120631 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:34.120607 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:37:34.120693 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:34.120676 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:37:34.120734 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:34.120711 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:37:34.122530 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:34.122497 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:37:34.122573 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:34.122544 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:37:34.122708 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:34.122572 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:37:34.968741 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:34.968699 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" event={"ID":"be1382e4-d585-4e8a-b5e5-f072ea92f29a","Type":"ContainerStarted","Data":"7711108f40cdeffda5b8640bd6980fb4aff1e6c4cb45a97e27ccf708f9a2955c"} Nov 04 05:37:34.969209 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:34.968767 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" event={"ID":"be1382e4-d585-4e8a-b5e5-f072ea92f29a","Type":"ContainerStarted","Data":"10b580275d36089dd8befa6a0b1f33d8537956aec265ffba565a0a5cafe6cbe3"} Nov 04 05:37:34.970303 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:34.970285 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" event={"ID":"52754caf-6be9-4c0f-a8f4-52c2aedf159e","Type":"ContainerStarted","Data":"3dbc0f5cb04d19d742e69a7f9e7bd8f43d04addd4ae9a7ba344b8344d12e5377"} Nov 04 05:37:34.970381 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:34.970318 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" event={"ID":"52754caf-6be9-4c0f-a8f4-52c2aedf159e","Type":"ContainerStarted","Data":"1055e520f1780cd163fb5bd3dcfc3c91de37be21a4033aa3faa1a5fc1bc2f997"} Nov 04 05:37:35.164641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:35.164579 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" podStartSLOduration=2.164564592 podStartE2EDuration="2.164564592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:37:35.153302223 +0000 UTC m=+625.154937945" watchObservedRunningTime="2026-11-04 05:37:35.164564592 +0000 UTC m=+625.166200317" Nov 04 05:37:35.300744 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:35.300081 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" podStartSLOduration=2.300059473 podStartE2EDuration="2.300059473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:37:35.297356639 +0000 UTC m=+625.298992366" watchObservedRunningTime="2026-11-04 05:37:35.300059473 +0000 UTC m=+625.301695193" Nov 04 05:37:35.663262 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:35.663234 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:35.664835 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:35.664811 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-t77lc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Nov 04 05:37:35.664943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:35.664854 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" podUID="be1382e4-d585-4e8a-b5e5-f072ea92f29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Nov 04 05:37:35.726688 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:35.724938 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:35.726688 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:35.725392 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-jrfvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Nov 04 05:37:35.726688 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:35.725548 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" podUID="52754caf-6be9-4c0f-a8f4-52c2aedf159e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Nov 04 05:37:36.663308 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:36.663272 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-t77lc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Nov 04 05:37:36.663748 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:36.663325 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" podUID="be1382e4-d585-4e8a-b5e5-f072ea92f29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Nov 04 05:37:36.718695 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:36.718141 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-jrfvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Nov 04 05:37:36.718695 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:36.718216 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" podUID="52754caf-6be9-4c0f-a8f4-52c2aedf159e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Nov 04 05:37:37.663134 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:37.663096 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-t77lc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Nov 04 05:37:37.663309 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:37.663149 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" podUID="be1382e4-d585-4e8a-b5e5-f072ea92f29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Nov 04 05:37:37.721691 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:37.719013 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-jrfvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Nov 04 05:37:37.721691 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:37.719074 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" podUID="52754caf-6be9-4c0f-a8f4-52c2aedf159e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Nov 04 05:37:38.663105 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:38.663067 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-t77lc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Nov 04 05:37:38.663271 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:38.663121 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" podUID="be1382e4-d585-4e8a-b5e5-f072ea92f29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Nov 04 05:37:38.719070 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:38.719026 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-jrfvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Nov 04 05:37:38.719323 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:38.719295 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" podUID="52754caf-6be9-4c0f-a8f4-52c2aedf159e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Nov 04 05:37:39.663469 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:39.663432 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-t77lc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Nov 04 05:37:39.663881 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:39.663491 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" podUID="be1382e4-d585-4e8a-b5e5-f072ea92f29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Nov 04 05:37:39.728687 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:39.728069 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-jrfvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Nov 04 05:37:39.728687 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:39.728147 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" podUID="52754caf-6be9-4c0f-a8f4-52c2aedf159e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Nov 04 05:37:40.057328 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:40.057257 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-79567948bc-gln67"] Nov 04 05:37:40.057479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:40.057459 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-79567948bc-gln67" podUID="7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8" containerName="manager" containerID="cri-o://e65248c3e7a4d1fd55ba403c32e8e34db450de01802b07e0ed36c3363797aa86" gracePeriod=30 Nov 04 05:37:40.332596 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:40.332573 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-79567948bc-gln67" Nov 04 05:37:40.474519 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:40.474478 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8-metrics-certs\") pod \"7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8\" (UID: \"7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8\") " Nov 04 05:37:40.474717 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:40.474565 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8-kube-api-access-l4p4v\" (UniqueName: \"kubernetes.io/projected/7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8-kube-api-access-l4p4v\") pod \"7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8\" (UID: \"7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8\") " Nov 04 05:37:40.477044 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:40.477009 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8-metrics-certs" pod "7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8" (UID: "7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:37:40.477479 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:40.477453 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8-kube-api-access-l4p4v" pod "7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8" (UID: "7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8"). InnerVolumeSpecName "kube-api-access-l4p4v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:37:40.575896 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:40.575810 4859 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8-metrics-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:37:40.575896 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:40.575839 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4p4v\" (UniqueName: \"kubernetes.io/projected/7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8-kube-api-access-l4p4v\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:37:40.663308 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:40.663273 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-t77lc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Nov 04 05:37:40.663480 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:40.663337 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" podUID="be1382e4-d585-4e8a-b5e5-f072ea92f29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Nov 04 05:37:40.716150 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:40.716118 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-jrfvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Nov 04 05:37:40.716530 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:40.716174 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" podUID="52754caf-6be9-4c0f-a8f4-52c2aedf159e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Nov 04 05:37:41.039643 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:41.039607 4859 generic.go:356] "Generic (PLEG): container finished" podID="7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8" containerID="e65248c3e7a4d1fd55ba403c32e8e34db450de01802b07e0ed36c3363797aa86" exitCode=0 Nov 04 05:37:41.039842 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:41.039690 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-79567948bc-gln67" Nov 04 05:37:41.039842 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:41.039709 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-79567948bc-gln67" event={"ID":"7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8","Type":"ContainerDied","Data":"e65248c3e7a4d1fd55ba403c32e8e34db450de01802b07e0ed36c3363797aa86"} Nov 04 05:37:41.039842 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:41.039788 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-79567948bc-gln67" event={"ID":"7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8","Type":"ContainerDied","Data":"ab1996d2fb2abdd75acb21dc792e3a709b2d4f726465bfcac5391d20c765787a"} Nov 04 05:37:41.039842 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:41.039822 4859 scope.go:122] "RemoveContainer" containerID="e65248c3e7a4d1fd55ba403c32e8e34db450de01802b07e0ed36c3363797aa86" Nov 04 05:37:41.048800 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:41.048781 4859 scope.go:122] "RemoveContainer" containerID="e65248c3e7a4d1fd55ba403c32e8e34db450de01802b07e0ed36c3363797aa86" Nov 04 05:37:41.049080 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:37:41.049057 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e65248c3e7a4d1fd55ba403c32e8e34db450de01802b07e0ed36c3363797aa86\": container with ID starting with e65248c3e7a4d1fd55ba403c32e8e34db450de01802b07e0ed36c3363797aa86 not found: ID does not exist" containerID="e65248c3e7a4d1fd55ba403c32e8e34db450de01802b07e0ed36c3363797aa86" Nov 04 05:37:41.049172 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:41.049086 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e65248c3e7a4d1fd55ba403c32e8e34db450de01802b07e0ed36c3363797aa86"} err="failed to get container status \"e65248c3e7a4d1fd55ba403c32e8e34db450de01802b07e0ed36c3363797aa86\": rpc error: code = NotFound desc = could not find container \"e65248c3e7a4d1fd55ba403c32e8e34db450de01802b07e0ed36c3363797aa86\": container with ID starting with e65248c3e7a4d1fd55ba403c32e8e34db450de01802b07e0ed36c3363797aa86 not found: ID does not exist" Nov 04 05:37:41.219169 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:41.219137 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-79567948bc-gln67"] Nov 04 05:37:41.273513 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:41.273482 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-79567948bc-gln67"] Nov 04 05:37:41.662811 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:41.662771 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-t77lc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Nov 04 05:37:41.662964 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:41.662838 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" podUID="be1382e4-d585-4e8a-b5e5-f072ea92f29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Nov 04 05:37:41.716295 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:41.716258 4859 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-jrfvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Nov 04 05:37:41.716641 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:41.716315 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" podUID="52754caf-6be9-4c0f-a8f4-52c2aedf159e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Nov 04 05:37:42.120782 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:42.120713 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8" path="/var/lib/kubelet/pods/7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8/volumes" Nov 04 05:37:42.892461 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:42.892431 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:42.892984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:42.892810 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:42.894048 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:42.894021 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:37:43.095824 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:43.095795 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:43.096235 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:43.096211 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:43.096959 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:37:43.096940 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-t77lc" Nov 04 05:37:58.341144 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:37:58.341107 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:38:28.341433 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:38:28.341395 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:38:36.727164 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.727129 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm"] Nov 04 05:38:36.727865 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.727845 4859 cpu_manager.go:414] "containerMap: removing container" podUID="7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8" containerName="manager" Nov 04 05:38:36.727958 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.727867 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8" containerName="manager" Nov 04 05:38:36.728057 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.727976 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f8e0fec-b7d5-43d7-b893-8b9f21beb7f8" containerName="manager" Nov 04 05:38:36.731208 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.731186 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.788263 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.788230 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-90dc8d-openshift-default-dockercfg-8rgkv\"" Nov 04 05:38:36.828887 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.828853 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-workload-socket\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.829060 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.828929 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-envoy\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.829060 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.828964 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-workload-certs\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.829060 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.828985 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-podinfo\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.829226 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.829065 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c684e324-0d8e-438d-b589-8a671f7c8b18-istiod-ca-cert\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.829226 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.829115 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-credential-socket\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.829226 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.829144 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsrt4\" (UniqueName: \"kubernetes.io/projected/c684e324-0d8e-438d-b589-8a671f7c8b18-kube-api-access-vsrt4\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.829377 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.829287 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-token\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.829377 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.829355 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-data\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.844247 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.844216 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm"] Nov 04 05:38:36.930001 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.929972 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-envoy\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.930001 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.930005 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-workload-certs\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.930241 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.930027 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-podinfo\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.930241 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.930148 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c684e324-0d8e-438d-b589-8a671f7c8b18-istiod-ca-cert\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.930241 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.930195 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-credential-socket\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.930402 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.930329 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsrt4\" (UniqueName: \"kubernetes.io/projected/c684e324-0d8e-438d-b589-8a671f7c8b18-kube-api-access-vsrt4\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.930467 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.930407 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-workload-certs\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.930467 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.930423 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-token\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.930573 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.930524 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-data\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.930573 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.930529 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-credential-socket\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.930707 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.930587 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-workload-socket\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.930831 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.930808 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-data\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.930966 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.930948 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-workload-socket\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.931019 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.930972 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c684e324-0d8e-438d-b589-8a671f7c8b18-istiod-ca-cert\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.932893 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.932875 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-envoy\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.933095 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.933078 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-podinfo\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.996130 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.996060 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-token\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:36.996261 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:36.996174 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsrt4\" (UniqueName: \"kubernetes.io/projected/c684e324-0d8e-438d-b589-8a671f7c8b18-kube-api-access-vsrt4\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:37.043654 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.043575 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:37.045943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.045919 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm"] Nov 04 05:38:37.049801 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.049785 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.106790 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.106762 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-098178-openshift-default-dockercfg-zfjds\"" Nov 04 05:38:37.127193 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.127166 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm"] Nov 04 05:38:37.133424 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.133391 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-credential-socket\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.133544 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.133469 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-workload-certs\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.133544 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.133501 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-workload-socket\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.133544 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.133533 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istiod-ca-cert\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.133711 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.133600 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-data\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.133711 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.133628 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwtjk\" (UniqueName: \"kubernetes.io/projected/6fda8c73-74bb-49ce-b03e-e5517015ee6f-kube-api-access-fwtjk\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.134137 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.133807 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-envoy\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.134137 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.133858 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-podinfo\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.134137 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.133897 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-token\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.238397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.236612 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-data\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.238397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.236677 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwtjk\" (UniqueName: \"kubernetes.io/projected/6fda8c73-74bb-49ce-b03e-e5517015ee6f-kube-api-access-fwtjk\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.238397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.236788 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-envoy\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.238397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.236825 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-podinfo\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.238397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.236861 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-token\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.238397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.236919 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-credential-socket\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.238397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.236955 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-workload-certs\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.238397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.236986 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-workload-socket\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.238397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.237020 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istiod-ca-cert\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.238397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.237135 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-data\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.238397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.237401 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-credential-socket\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.238397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.237698 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-workload-certs\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.238397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.237815 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istiod-ca-cert\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.238397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.238230 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-workload-socket\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.241030 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.240965 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-podinfo\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.244470 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.244447 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-envoy\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.303122 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.303050 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-token\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.303239 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.303126 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwtjk\" (UniqueName: \"kubernetes.io/projected/6fda8c73-74bb-49ce-b03e-e5517015ee6f-kube-api-access-fwtjk\") pod \"e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.375399 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.375365 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:37.509411 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.509369 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm"] Nov 04 05:38:37.513261 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:38:37.513234 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc684e324_0d8e_438d_b589_8a671f7c8b18.slice/crio-d94b1c4c8820ae6d52661b019e3e93d4d6b66c34da5e07997a1df07bcd2ab0c4 WatchSource:0}: Error finding container d94b1c4c8820ae6d52661b019e3e93d4d6b66c34da5e07997a1df07bcd2ab0c4: Status 404 returned error can't find the container with id d94b1c4c8820ae6d52661b019e3e93d4d6b66c34da5e07997a1df07bcd2ab0c4 Nov 04 05:38:37.515486 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.515454 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:38:37.515575 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.515518 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:38:37.515575 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.515549 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:38:37.798519 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.798486 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm"] Nov 04 05:38:37.802012 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:38:37.801980 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fda8c73_74bb_49ce_b03e_e5517015ee6f.slice/crio-e94abd934f264eed8241a021523f40968b6a74f69e3e09b8583c6b8c41a19b1c WatchSource:0}: Error finding container e94abd934f264eed8241a021523f40968b6a74f69e3e09b8583c6b8c41a19b1c: Status 404 returned error can't find the container with id e94abd934f264eed8241a021523f40968b6a74f69e3e09b8583c6b8c41a19b1c Nov 04 05:38:37.804483 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.804456 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:38:37.804549 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.804516 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:38:37.804549 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:37.804544 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:38:38.326344 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:38.326077 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" event={"ID":"6fda8c73-74bb-49ce-b03e-e5517015ee6f","Type":"ContainerStarted","Data":"a8d30f9bcd6b72a9f1b3fbce18e5a4e71d37e61b5e446510c32d858d4628876b"} Nov 04 05:38:38.326344 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:38.326128 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" event={"ID":"6fda8c73-74bb-49ce-b03e-e5517015ee6f","Type":"ContainerStarted","Data":"e94abd934f264eed8241a021523f40968b6a74f69e3e09b8583c6b8c41a19b1c"} Nov 04 05:38:38.328220 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:38.328019 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" event={"ID":"c684e324-0d8e-438d-b589-8a671f7c8b18","Type":"ContainerStarted","Data":"513229f772e0aad21292d0ad06222275823360bb94f81f2952b71caea3516569"} Nov 04 05:38:38.328220 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:38.328064 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" event={"ID":"c684e324-0d8e-438d-b589-8a671f7c8b18","Type":"ContainerStarted","Data":"d94b1c4c8820ae6d52661b019e3e93d4d6b66c34da5e07997a1df07bcd2ab0c4"} Nov 04 05:38:38.376194 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:38.376153 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:38.381694 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:38.381643 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:38.458328 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:38.458274 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" podStartSLOduration=2.458259212 podStartE2EDuration="2.458259212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:38.457914535 +0000 UTC m=+688.459550260" watchObservedRunningTime="2026-11-04 05:38:38.458259212 +0000 UTC m=+688.459894936" Nov 04 05:38:38.723377 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:38.723325 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" podStartSLOduration=2.723311414 podStartE2EDuration="2.723311414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:38.722363921 +0000 UTC m=+688.723999647" watchObservedRunningTime="2026-11-04 05:38:38.723311414 +0000 UTC m=+688.724947140" Nov 04 05:38:39.044007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:39.043919 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:39.049939 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:39.049911 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:39.333877 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:39.333793 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:39.333877 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:39.333855 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:39.334574 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:39.334554 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:38:39.334729 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:39.334709 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:38:46.366559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:46.366521 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj"] Nov 04 05:38:46.371688 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:46.371652 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" Nov 04 05:38:46.431396 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:46.431369 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-fbbad9-dockercfg-d4s4t\"" Nov 04 05:38:46.448438 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:46.448401 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krgc6\" (UniqueName: \"kubernetes.io/projected/082c0bec-9706-4234-8ce4-e57f08aa23f5-kube-api-access-krgc6\") pod \"payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj\" (UID: \"082c0bec-9706-4234-8ce4-e57f08aa23f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" Nov 04 05:38:46.448596 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:46.448457 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/082c0bec-9706-4234-8ce4-e57f08aa23f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj\" (UID: \"082c0bec-9706-4234-8ce4-e57f08aa23f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" Nov 04 05:38:46.495604 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:46.495569 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-fbbad9\"" Nov 04 05:38:46.497508 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:46.497484 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj"] Nov 04 05:38:46.549894 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:46.549863 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krgc6\" (UniqueName: \"kubernetes.io/projected/082c0bec-9706-4234-8ce4-e57f08aa23f5-kube-api-access-krgc6\") pod \"payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj\" (UID: \"082c0bec-9706-4234-8ce4-e57f08aa23f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" Nov 04 05:38:46.550069 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:46.549925 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/082c0bec-9706-4234-8ce4-e57f08aa23f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj\" (UID: \"082c0bec-9706-4234-8ce4-e57f08aa23f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" Nov 04 05:38:46.550728 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:46.550702 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/082c0bec-9706-4234-8ce4-e57f08aa23f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj\" (UID: \"082c0bec-9706-4234-8ce4-e57f08aa23f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" Nov 04 05:38:46.615584 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:46.615551 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krgc6\" (UniqueName: \"kubernetes.io/projected/082c0bec-9706-4234-8ce4-e57f08aa23f5-kube-api-access-krgc6\") pod \"payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj\" (UID: \"082c0bec-9706-4234-8ce4-e57f08aa23f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" Nov 04 05:38:46.685140 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:46.685112 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" Nov 04 05:38:47.072496 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:47.072414 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj"] Nov 04 05:38:47.076171 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:38:47.076144 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod082c0bec_9706_4234_8ce4_e57f08aa23f5.slice/crio-ec4b403540265404762a1bf0f8ec8372e6275447fa88298a6d4078099da38c54 WatchSource:0}: Error finding container ec4b403540265404762a1bf0f8ec8372e6275447fa88298a6d4078099da38c54: Status 404 returned error can't find the container with id ec4b403540265404762a1bf0f8ec8372e6275447fa88298a6d4078099da38c54 Nov 04 05:38:47.377919 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:47.377825 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" event={"ID":"082c0bec-9706-4234-8ce4-e57f08aa23f5","Type":"ContainerStarted","Data":"13d852f8e60388b7dc548e556681d6fa5d09f80bf28c08f82594b6afad1b3b8e"} Nov 04 05:38:47.378384 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:47.377923 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" Nov 04 05:38:47.378384 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:47.377944 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" event={"ID":"082c0bec-9706-4234-8ce4-e57f08aa23f5","Type":"ContainerStarted","Data":"ec4b403540265404762a1bf0f8ec8372e6275447fa88298a6d4078099da38c54"} Nov 04 05:38:47.504640 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:47.504584 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" podStartSLOduration=1.504569079 podStartE2EDuration="1.504569079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:47.50225996 +0000 UTC m=+697.503895685" watchObservedRunningTime="2026-11-04 05:38:47.504569079 +0000 UTC m=+697.506204803" Nov 04 05:38:47.826420 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:47.826385 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx"] Nov 04 05:38:47.830037 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:47.830021 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:47.886809 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:47.886781 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-a84d48-serving-cert\"" Nov 04 05:38:47.893306 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:47.893277 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx"] Nov 04 05:38:47.962305 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:47.962274 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-maas-api-tls\") pod \"maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:47.962536 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:47.962333 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-metrics-certs\") pod \"maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:47.962536 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:47.962413 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ftrn\" (UniqueName: \"kubernetes.io/projected/4645be22-7060-4e20-9fde-6f7595ef45da-kube-api-access-5ftrn\") pod \"maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:47.962536 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:47.962488 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4645be22-7060-4e20-9fde-6f7595ef45da-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:48.063591 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:48.063560 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-maas-api-tls\") pod \"maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:48.063778 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:48.063619 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-metrics-certs\") pod \"maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:48.063778 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:48.063643 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ftrn\" (UniqueName: \"kubernetes.io/projected/4645be22-7060-4e20-9fde-6f7595ef45da-kube-api-access-5ftrn\") pod \"maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:48.063778 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:48.063710 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4645be22-7060-4e20-9fde-6f7595ef45da-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:48.063778 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:38:48.063744 4859 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-serving-cert: secret "maas-api-e2e-worker-w3-a84d48-serving-cert" not found Nov 04 05:38:48.063921 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:38:48.063839 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-maas-api-tls podName:4645be22-7060-4e20-9fde-6f7595ef45da nodeName:}" failed. No retries permitted until 2026-11-04 05:38:48.563818018 +0000 UTC m=+698.565453741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-maas-api-tls") pod "maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" (UID: "4645be22-7060-4e20-9fde-6f7595ef45da") : secret "maas-api-e2e-worker-w3-a84d48-serving-cert" not found Nov 04 05:38:48.064379 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:48.064360 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4645be22-7060-4e20-9fde-6f7595ef45da-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:48.066112 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:48.066090 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-metrics-certs\") pod \"maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:48.131714 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:48.131638 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ftrn\" (UniqueName: \"kubernetes.io/projected/4645be22-7060-4e20-9fde-6f7595ef45da-kube-api-access-5ftrn\") pod \"maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:48.571482 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:48.571440 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-maas-api-tls\") pod \"maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:48.574289 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:48.574259 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-maas-api-tls\") pod \"maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:48.746324 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:48.746293 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:49.133143 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:49.133103 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx"] Nov 04 05:38:49.137244 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:38:49.137215 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4645be22_7060_4e20_9fde_6f7595ef45da.slice/crio-2bc0846e398da387ab80499c9f1c39e06b724db5557dfa553a7745ca1941a975 WatchSource:0}: Error finding container 2bc0846e398da387ab80499c9f1c39e06b724db5557dfa553a7745ca1941a975: Status 404 returned error can't find the container with id 2bc0846e398da387ab80499c9f1c39e06b724db5557dfa553a7745ca1941a975 Nov 04 05:38:49.139076 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:49.139055 4859 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:38:49.389159 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:49.389077 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" event={"ID":"4645be22-7060-4e20-9fde-6f7595ef45da","Type":"ContainerStarted","Data":"2bc0846e398da387ab80499c9f1c39e06b724db5557dfa553a7745ca1941a975"} Nov 04 05:38:51.418771 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.418729 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" event={"ID":"4645be22-7060-4e20-9fde-6f7595ef45da","Type":"ContainerStarted","Data":"9e8a31add99f872d7fe607f1918fad6a8c9432c517f7797aefcf6358a416f105"} Nov 04 05:38:51.420321 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.420297 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:51.547617 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.547566 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" podStartSLOduration=2.7344156870000003 podStartE2EDuration="4.547553925s" totalImagesPullingTime="1.813138238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:51.546315035 +0000 UTC m=+701.547950763" watchObservedRunningTime="2026-11-04 05:38:51.547553925 +0000 UTC m=+701.549189650" Nov 04 05:38:51.691105 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.691031 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l"] Nov 04 05:38:51.694794 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.694776 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" Nov 04 05:38:51.754335 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.754309 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-a84d48\"" Nov 04 05:38:51.754477 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.754381 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-a84d48-dockercfg-vdz68\"" Nov 04 05:38:51.757643 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.757618 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l"] Nov 04 05:38:51.828407 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.828364 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74964f80-4702-460f-95bd-864bd6b30d93-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l\" (UID: \"74964f80-4702-460f-95bd-864bd6b30d93\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" Nov 04 05:38:51.828805 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.828780 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7swfz\" (UniqueName: \"kubernetes.io/projected/74964f80-4702-460f-95bd-864bd6b30d93-kube-api-access-7swfz\") pod \"payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l\" (UID: \"74964f80-4702-460f-95bd-864bd6b30d93\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" Nov 04 05:38:51.831879 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.831828 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746"] Nov 04 05:38:51.845189 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.844004 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" Nov 04 05:38:51.888987 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.888954 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746"] Nov 04 05:38:51.929683 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.929634 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8fnp\" (UniqueName: \"kubernetes.io/projected/3567dc66-3594-4163-a4d1-e2a4e9d5b408-kube-api-access-b8fnp\") pod \"payload-processing-e2e-worker-w3-a84d48-55855658b-sl746\" (UID: \"3567dc66-3594-4163-a4d1-e2a4e9d5b408\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" Nov 04 05:38:51.929824 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.929806 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74964f80-4702-460f-95bd-864bd6b30d93-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l\" (UID: \"74964f80-4702-460f-95bd-864bd6b30d93\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" Nov 04 05:38:51.929870 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.929855 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3567dc66-3594-4163-a4d1-e2a4e9d5b408-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-a84d48-55855658b-sl746\" (UID: \"3567dc66-3594-4163-a4d1-e2a4e9d5b408\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" Nov 04 05:38:51.929920 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.929879 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7swfz\" (UniqueName: \"kubernetes.io/projected/74964f80-4702-460f-95bd-864bd6b30d93-kube-api-access-7swfz\") pod \"payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l\" (UID: \"74964f80-4702-460f-95bd-864bd6b30d93\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" Nov 04 05:38:51.930560 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.930542 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74964f80-4702-460f-95bd-864bd6b30d93-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l\" (UID: \"74964f80-4702-460f-95bd-864bd6b30d93\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" Nov 04 05:38:51.995727 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:51.995619 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7swfz\" (UniqueName: \"kubernetes.io/projected/74964f80-4702-460f-95bd-864bd6b30d93-kube-api-access-7swfz\") pod \"payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l\" (UID: \"74964f80-4702-460f-95bd-864bd6b30d93\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" Nov 04 05:38:52.008551 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:52.008522 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" Nov 04 05:38:52.030795 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:52.030765 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3567dc66-3594-4163-a4d1-e2a4e9d5b408-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-a84d48-55855658b-sl746\" (UID: \"3567dc66-3594-4163-a4d1-e2a4e9d5b408\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" Nov 04 05:38:52.030913 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:52.030841 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8fnp\" (UniqueName: \"kubernetes.io/projected/3567dc66-3594-4163-a4d1-e2a4e9d5b408-kube-api-access-b8fnp\") pod \"payload-processing-e2e-worker-w3-a84d48-55855658b-sl746\" (UID: \"3567dc66-3594-4163-a4d1-e2a4e9d5b408\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" Nov 04 05:38:52.031445 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:52.031423 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3567dc66-3594-4163-a4d1-e2a4e9d5b408-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-a84d48-55855658b-sl746\" (UID: \"3567dc66-3594-4163-a4d1-e2a4e9d5b408\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" Nov 04 05:38:52.098081 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:52.098048 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8fnp\" (UniqueName: \"kubernetes.io/projected/3567dc66-3594-4163-a4d1-e2a4e9d5b408-kube-api-access-b8fnp\") pod \"payload-processing-e2e-worker-w3-a84d48-55855658b-sl746\" (UID: \"3567dc66-3594-4163-a4d1-e2a4e9d5b408\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" Nov 04 05:38:52.163825 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:52.163795 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" Nov 04 05:38:52.395539 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:52.395455 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l"] Nov 04 05:38:52.398974 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:38:52.398950 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74964f80_4702_460f_95bd_864bd6b30d93.slice/crio-000395a6180f4a5c5f6b33fc99079df919160d4cbdb5362ff2a5fcc0b865d018 WatchSource:0}: Error finding container 000395a6180f4a5c5f6b33fc99079df919160d4cbdb5362ff2a5fcc0b865d018: Status 404 returned error can't find the container with id 000395a6180f4a5c5f6b33fc99079df919160d4cbdb5362ff2a5fcc0b865d018 Nov 04 05:38:52.424802 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:52.424770 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" event={"ID":"74964f80-4702-460f-95bd-864bd6b30d93","Type":"ContainerStarted","Data":"000395a6180f4a5c5f6b33fc99079df919160d4cbdb5362ff2a5fcc0b865d018"} Nov 04 05:38:52.550008 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:52.549977 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746"] Nov 04 05:38:52.553678 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:38:52.553643 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3567dc66_3594_4163_a4d1_e2a4e9d5b408.slice/crio-72ab87ebc13aa6d1c683cf63187cd7a80e5dfddb257149b59e519a0ef42db4cd WatchSource:0}: Error finding container 72ab87ebc13aa6d1c683cf63187cd7a80e5dfddb257149b59e519a0ef42db4cd: Status 404 returned error can't find the container with id 72ab87ebc13aa6d1c683cf63187cd7a80e5dfddb257149b59e519a0ef42db4cd Nov 04 05:38:53.431807 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.431766 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" event={"ID":"74964f80-4702-460f-95bd-864bd6b30d93","Type":"ContainerStarted","Data":"0408db1f14fccd416dd3e51d7646370b1dd402c89da0abec345839d03078b8de"} Nov 04 05:38:53.432230 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.431965 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" Nov 04 05:38:53.433251 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.433229 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" event={"ID":"3567dc66-3594-4163-a4d1-e2a4e9d5b408","Type":"ContainerStarted","Data":"3db1fa2a750118eceabe9cc94366cd6d10cc21732de992558a3b07c2f4b257b6"} Nov 04 05:38:53.433374 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.433266 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" event={"ID":"3567dc66-3594-4163-a4d1-e2a4e9d5b408","Type":"ContainerStarted","Data":"72ab87ebc13aa6d1c683cf63187cd7a80e5dfddb257149b59e519a0ef42db4cd"} Nov 04 05:38:53.433374 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.433355 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" Nov 04 05:38:53.558751 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.558697 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" podStartSLOduration=2.558684332 podStartE2EDuration="2.558684332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:53.556296333 +0000 UTC m=+703.557932059" watchObservedRunningTime="2026-11-04 05:38:53.558684332 +0000 UTC m=+703.560320078" Nov 04 05:38:53.625120 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.625086 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p"] Nov 04 05:38:53.628936 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.628916 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:38:53.687719 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.687601 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-901a00-serving-cert\"" Nov 04 05:38:53.688233 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.688098 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" podStartSLOduration=2.688079934 podStartE2EDuration="2.688079934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:53.685522273 +0000 UTC m=+703.687157999" watchObservedRunningTime="2026-11-04 05:38:53.688079934 +0000 UTC m=+703.689715666" Nov 04 05:38:53.696128 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.694889 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p"] Nov 04 05:38:53.747617 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.747585 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-maas-api-tls\") pod \"maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:38:53.747815 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.747624 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6446e2c-33aa-434c-b167-8c69862baaa9-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:38:53.747815 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.747799 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-metrics-certs\") pod \"maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:38:53.747934 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.747838 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrnfh\" (UniqueName: \"kubernetes.io/projected/e6446e2c-33aa-434c-b167-8c69862baaa9-kube-api-access-xrnfh\") pod \"maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:38:53.849347 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.849314 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrnfh\" (UniqueName: \"kubernetes.io/projected/e6446e2c-33aa-434c-b167-8c69862baaa9-kube-api-access-xrnfh\") pod \"maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:38:53.849516 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.849395 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-maas-api-tls\") pod \"maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:38:53.849516 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.849415 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6446e2c-33aa-434c-b167-8c69862baaa9-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:38:53.849516 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.849496 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-metrics-certs\") pod \"maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:38:53.849762 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:38:53.849548 4859 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-serving-cert: secret "maas-api-e2e-worker-w4-901a00-serving-cert" not found Nov 04 05:38:53.849762 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:38:53.849621 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-maas-api-tls podName:e6446e2c-33aa-434c-b167-8c69862baaa9 nodeName:}" failed. No retries permitted until 2026-11-04 05:38:54.349598389 +0000 UTC m=+704.351234110 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-maas-api-tls") pod "maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" (UID: "e6446e2c-33aa-434c-b167-8c69862baaa9") : secret "maas-api-e2e-worker-w4-901a00-serving-cert" not found Nov 04 05:38:53.850204 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.850180 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6446e2c-33aa-434c-b167-8c69862baaa9-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:38:53.851921 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.851899 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-metrics-certs\") pod \"maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:38:53.934164 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:53.933818 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrnfh\" (UniqueName: \"kubernetes.io/projected/e6446e2c-33aa-434c-b167-8c69862baaa9-kube-api-access-xrnfh\") pod \"maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:38:54.358713 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:54.358649 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-maas-api-tls\") pod \"maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:38:54.361288 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:54.361257 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-maas-api-tls\") pod \"maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:38:54.542493 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:54.542458 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:38:54.934387 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:54.934230 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p"] Nov 04 05:38:54.937519 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:38:54.937487 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6446e2c_33aa_434c_b167_8c69862baaa9.slice/crio-0dcfeb2c30842ad69040a0cbc02fd9eb31ad9e68aeb064b8732ccf2994a4be63 WatchSource:0}: Error finding container 0dcfeb2c30842ad69040a0cbc02fd9eb31ad9e68aeb064b8732ccf2994a4be63: Status 404 returned error can't find the container with id 0dcfeb2c30842ad69040a0cbc02fd9eb31ad9e68aeb064b8732ccf2994a4be63 Nov 04 05:38:55.446063 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:55.446030 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" event={"ID":"e6446e2c-33aa-434c-b167-8c69862baaa9","Type":"ContainerStarted","Data":"0dcfeb2c30842ad69040a0cbc02fd9eb31ad9e68aeb064b8732ccf2994a4be63"} Nov 04 05:38:56.453206 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:56.453174 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" event={"ID":"e6446e2c-33aa-434c-b167-8c69862baaa9","Type":"ContainerStarted","Data":"f5b47c57a3e13ab88bd110f3c07d92120e832bb7de303721747eb60516ead368"} Nov 04 05:38:56.453647 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:56.453258 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:38:56.581578 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:56.581518 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" podStartSLOduration=2.554502626 podStartE2EDuration="3.581498916s" totalImagesPullingTime="1.02699629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:56.579597252 +0000 UTC m=+706.581232978" watchObservedRunningTime="2026-11-04 05:38:56.581498916 +0000 UTC m=+706.583134698" Nov 04 05:38:58.342006 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:38:58.341968 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:38:58.386581 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:58.386548 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" Nov 04 05:38:58.439543 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:58.439511 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:38:59.423492 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:59.423452 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2"] Nov 04 05:38:59.427390 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:59.427374 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:38:59.501821 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:59.501789 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-961485-serving-cert\"" Nov 04 05:38:59.515633 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:59.515599 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-metrics-certs\") pod \"maas-api-e2e-worker-w2-961485-5d6988956d-crlb2\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:38:59.515896 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:59.515862 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-maas-api-tls\") pod \"maas-api-e2e-worker-w2-961485-5d6988956d-crlb2\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:38:59.516144 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:59.516110 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wzcv\" (UniqueName: \"kubernetes.io/projected/f952eaf6-6354-4094-9a30-3c032183dfed-kube-api-access-6wzcv\") pod \"maas-api-e2e-worker-w2-961485-5d6988956d-crlb2\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:38:59.516302 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:59.516283 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f952eaf6-6354-4094-9a30-3c032183dfed-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-961485-5d6988956d-crlb2\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:38:59.541397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:59.541373 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2"] Nov 04 05:38:59.617815 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:59.617785 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f952eaf6-6354-4094-9a30-3c032183dfed-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-961485-5d6988956d-crlb2\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:38:59.617962 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:59.617831 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-metrics-certs\") pod \"maas-api-e2e-worker-w2-961485-5d6988956d-crlb2\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:38:59.617962 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:59.617947 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-maas-api-tls\") pod \"maas-api-e2e-worker-w2-961485-5d6988956d-crlb2\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:38:59.618036 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:59.617966 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wzcv\" (UniqueName: \"kubernetes.io/projected/f952eaf6-6354-4094-9a30-3c032183dfed-kube-api-access-6wzcv\") pod \"maas-api-e2e-worker-w2-961485-5d6988956d-crlb2\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:38:59.618099 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:38:59.618079 4859 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-serving-cert: secret "maas-api-e2e-worker-w2-961485-serving-cert" not found Nov 04 05:38:59.618167 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:38:59.618155 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-maas-api-tls podName:f952eaf6-6354-4094-9a30-3c032183dfed nodeName:}" failed. No retries permitted until 2026-11-04 05:39:00.118134141 +0000 UTC m=+710.119769845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-maas-api-tls") pod "maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" (UID: "f952eaf6-6354-4094-9a30-3c032183dfed") : secret "maas-api-e2e-worker-w2-961485-serving-cert" not found Nov 04 05:38:59.618423 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:59.618402 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f952eaf6-6354-4094-9a30-3c032183dfed-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-961485-5d6988956d-crlb2\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:38:59.620233 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:59.620211 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-metrics-certs\") pod \"maas-api-e2e-worker-w2-961485-5d6988956d-crlb2\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:38:59.683578 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:38:59.683513 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wzcv\" (UniqueName: \"kubernetes.io/projected/f952eaf6-6354-4094-9a30-3c032183dfed-kube-api-access-6wzcv\") pod \"maas-api-e2e-worker-w2-961485-5d6988956d-crlb2\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:39:00.126455 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:00.126382 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-maas-api-tls\") pod \"maas-api-e2e-worker-w2-961485-5d6988956d-crlb2\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:39:00.128767 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:00.128740 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-maas-api-tls\") pod \"maas-api-e2e-worker-w2-961485-5d6988956d-crlb2\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:39:00.340101 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:00.340071 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:39:00.733871 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:00.733837 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2"] Nov 04 05:39:00.737327 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:39:00.737298 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf952eaf6_6354_4094_9a30_3c032183dfed.slice/crio-1b8ccca54a84819c8c427862835c0995efcbef4a43348d5f0805fda5d11c7bb2 WatchSource:0}: Error finding container 1b8ccca54a84819c8c427862835c0995efcbef4a43348d5f0805fda5d11c7bb2: Status 404 returned error can't find the container with id 1b8ccca54a84819c8c427862835c0995efcbef4a43348d5f0805fda5d11c7bb2 Nov 04 05:39:01.499418 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:01.499382 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" event={"ID":"f952eaf6-6354-4094-9a30-3c032183dfed","Type":"ContainerStarted","Data":"1b8ccca54a84819c8c427862835c0995efcbef4a43348d5f0805fda5d11c7bb2"} Nov 04 05:39:02.464515 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:02.464489 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:39:02.506019 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:02.505982 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" event={"ID":"f952eaf6-6354-4094-9a30-3c032183dfed","Type":"ContainerStarted","Data":"c5e1422673fe31a827e18c3221b6749e00f611d48204825eda30a78665682253"} Nov 04 05:39:02.506167 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:02.506064 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:39:02.720008 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:02.719904 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" podStartSLOduration=2.8143010029999997 podStartE2EDuration="3.719888554s" totalImagesPullingTime="905.587551ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:39:02.717816622 +0000 UTC m=+712.719452359" watchObservedRunningTime="2026-11-04 05:39:02.719888554 +0000 UTC m=+712.721524319" Nov 04 05:39:04.442684 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:04.442642 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" Nov 04 05:39:04.443085 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:04.442725 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" Nov 04 05:39:07.908728 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:07.908692 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f"] Nov 04 05:39:07.913026 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:07.913005 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" Nov 04 05:39:08.033026 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.032987 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-961485-dockercfg-cmgc6\"" Nov 04 05:39:08.033367 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.033333 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-961485\"" Nov 04 05:39:08.034083 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.033626 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw4xk\" (UniqueName: \"kubernetes.io/projected/258acbd4-6cde-4fc3-b34a-da6e04cdb963-kube-api-access-bw4xk\") pod \"payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f\" (UID: \"258acbd4-6cde-4fc3-b34a-da6e04cdb963\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" Nov 04 05:39:08.034980 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.034937 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/258acbd4-6cde-4fc3-b34a-da6e04cdb963-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f\" (UID: \"258acbd4-6cde-4fc3-b34a-da6e04cdb963\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" Nov 04 05:39:08.037989 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.037106 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f"] Nov 04 05:39:08.089691 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.089640 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8"] Nov 04 05:39:08.093582 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.093564 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" Nov 04 05:39:08.136416 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.136382 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/258acbd4-6cde-4fc3-b34a-da6e04cdb963-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f\" (UID: \"258acbd4-6cde-4fc3-b34a-da6e04cdb963\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" Nov 04 05:39:08.136587 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.136493 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw4xk\" (UniqueName: \"kubernetes.io/projected/258acbd4-6cde-4fc3-b34a-da6e04cdb963-kube-api-access-bw4xk\") pod \"payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f\" (UID: \"258acbd4-6cde-4fc3-b34a-da6e04cdb963\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" Nov 04 05:39:08.137156 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.137133 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/258acbd4-6cde-4fc3-b34a-da6e04cdb963-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f\" (UID: \"258acbd4-6cde-4fc3-b34a-da6e04cdb963\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" Nov 04 05:39:08.161466 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.161392 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8"] Nov 04 05:39:08.202212 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.202180 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw4xk\" (UniqueName: \"kubernetes.io/projected/258acbd4-6cde-4fc3-b34a-da6e04cdb963-kube-api-access-bw4xk\") pod \"payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f\" (UID: \"258acbd4-6cde-4fc3-b34a-da6e04cdb963\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" Nov 04 05:39:08.226164 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.226128 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" Nov 04 05:39:08.237651 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.237617 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh4nc\" (UniqueName: \"kubernetes.io/projected/6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0-kube-api-access-gh4nc\") pod \"payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8\" (UID: \"6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" Nov 04 05:39:08.237864 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.237810 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8\" (UID: \"6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" Nov 04 05:39:08.339814 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.339781 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8\" (UID: \"6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" Nov 04 05:39:08.339991 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.339922 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh4nc\" (UniqueName: \"kubernetes.io/projected/6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0-kube-api-access-gh4nc\") pod \"payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8\" (UID: \"6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" Nov 04 05:39:08.340769 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.340745 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8\" (UID: \"6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" Nov 04 05:39:08.405036 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.404999 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh4nc\" (UniqueName: \"kubernetes.io/projected/6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0-kube-api-access-gh4nc\") pod \"payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8\" (UID: \"6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" Nov 04 05:39:08.406814 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.406794 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" Nov 04 05:39:08.517976 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.517950 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:39:08.634000 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.633962 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f"] Nov 04 05:39:08.642149 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:39:08.642113 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod258acbd4_6cde_4fc3_b34a_da6e04cdb963.slice/crio-a1329ef0071d5164b1542d7935a94491c2ea8b91713f826e7a46562eea6c75d9 WatchSource:0}: Error finding container a1329ef0071d5164b1542d7935a94491c2ea8b91713f826e7a46562eea6c75d9: Status 404 returned error can't find the container with id a1329ef0071d5164b1542d7935a94491c2ea8b91713f826e7a46562eea6c75d9 Nov 04 05:39:08.796353 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:08.796316 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8"] Nov 04 05:39:08.800012 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:39:08.799982 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9bc5b2_a5c8_47bd_a478_d40f2e11ccf0.slice/crio-789f86c4af560671fa1bb205220fd3fcb18a97868e2a084f33b1de9b66810778 WatchSource:0}: Error finding container 789f86c4af560671fa1bb205220fd3fcb18a97868e2a084f33b1de9b66810778: Status 404 returned error can't find the container with id 789f86c4af560671fa1bb205220fd3fcb18a97868e2a084f33b1de9b66810778 Nov 04 05:39:09.554315 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:09.554274 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" event={"ID":"258acbd4-6cde-4fc3-b34a-da6e04cdb963","Type":"ContainerStarted","Data":"32b3e484d189ce9728e35d91cd4b8cf8411bf2462da62a9a2bd44479014ecbc6"} Nov 04 05:39:09.554838 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:09.554337 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" event={"ID":"258acbd4-6cde-4fc3-b34a-da6e04cdb963","Type":"ContainerStarted","Data":"a1329ef0071d5164b1542d7935a94491c2ea8b91713f826e7a46562eea6c75d9"} Nov 04 05:39:09.554838 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:09.554474 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" Nov 04 05:39:09.556137 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:09.556111 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" event={"ID":"6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0","Type":"ContainerStarted","Data":"c6f4dfca622bb2bdd2a087412a33a01f51dfaf9eaa2f550e0de8a94d75050029"} Nov 04 05:39:09.556303 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:09.556172 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" Nov 04 05:39:09.556303 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:09.556206 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" event={"ID":"6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0","Type":"ContainerStarted","Data":"789f86c4af560671fa1bb205220fd3fcb18a97868e2a084f33b1de9b66810778"} Nov 04 05:39:09.679051 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:09.678995 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" podStartSLOduration=2.678981935 podStartE2EDuration="2.678981935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:39:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:39:09.678059521 +0000 UTC m=+719.679695247" watchObservedRunningTime="2026-11-04 05:39:09.678981935 +0000 UTC m=+719.680617657" Nov 04 05:39:09.811266 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:09.809463 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" podStartSLOduration=1.8094416070000001 podStartE2EDuration="1.809441607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:39:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:39:09.80912328 +0000 UTC m=+719.810759005" watchObservedRunningTime="2026-11-04 05:39:09.809441607 +0000 UTC m=+719.811077334" Nov 04 05:39:10.746885 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:10.746850 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f"] Nov 04 05:39:10.751718 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:10.751702 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" Nov 04 05:39:10.813013 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:10.812980 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f"] Nov 04 05:39:10.869180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:10.868446 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-283815\"" Nov 04 05:39:10.869180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:10.886293 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27f04fa4-41f7-4988-b5a4-d81187643cf7-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f\" (UID: \"27f04fa4-41f7-4988-b5a4-d81187643cf7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" Nov 04 05:39:10.869180 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:10.886404 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ltx7\" (UniqueName: \"kubernetes.io/projected/27f04fa4-41f7-4988-b5a4-d81187643cf7-kube-api-access-8ltx7\") pod \"payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f\" (UID: \"27f04fa4-41f7-4988-b5a4-d81187643cf7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" Nov 04 05:39:10.987933 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:10.987891 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ltx7\" (UniqueName: \"kubernetes.io/projected/27f04fa4-41f7-4988-b5a4-d81187643cf7-kube-api-access-8ltx7\") pod \"payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f\" (UID: \"27f04fa4-41f7-4988-b5a4-d81187643cf7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" Nov 04 05:39:10.988120 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:10.988058 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27f04fa4-41f7-4988-b5a4-d81187643cf7-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f\" (UID: \"27f04fa4-41f7-4988-b5a4-d81187643cf7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" Nov 04 05:39:10.988771 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:10.988753 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27f04fa4-41f7-4988-b5a4-d81187643cf7-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f\" (UID: \"27f04fa4-41f7-4988-b5a4-d81187643cf7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" Nov 04 05:39:11.053508 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:11.053426 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ltx7\" (UniqueName: \"kubernetes.io/projected/27f04fa4-41f7-4988-b5a4-d81187643cf7-kube-api-access-8ltx7\") pod \"payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f\" (UID: \"27f04fa4-41f7-4988-b5a4-d81187643cf7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" Nov 04 05:39:11.065450 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:11.065422 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" Nov 04 05:39:11.457692 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:11.454824 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f"] Nov 04 05:39:11.568842 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:11.568809 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" event={"ID":"27f04fa4-41f7-4988-b5a4-d81187643cf7","Type":"ContainerStarted","Data":"1cc40cae53d8feb15552a5ed5847eda217a1c90c06c65d752cd5a135b111a7d5"} Nov 04 05:39:11.568951 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:11.568865 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" event={"ID":"27f04fa4-41f7-4988-b5a4-d81187643cf7","Type":"ContainerStarted","Data":"e93094368c6d95bdecbc9cdbe0d93ae5a25d010a4d398571319c60dbdacea941"} Nov 04 05:39:11.568951 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:11.568895 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" Nov 04 05:39:11.693181 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:11.693132 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" podStartSLOduration=1.6931190379999999 podStartE2EDuration="1.693119038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:39:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:39:11.692548583 +0000 UTC m=+721.694184307" watchObservedRunningTime="2026-11-04 05:39:11.693119038 +0000 UTC m=+721.694754763" Nov 04 05:39:16.634412 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:16.634380 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr"] Nov 04 05:39:16.639377 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:16.639358 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:16.698405 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:16.698380 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-90dc8d-serving-cert\"" Nov 04 05:39:16.755879 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:16.755833 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr"] Nov 04 05:39:16.756330 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:16.756306 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22dcf276-7849-4247-89d2-98a53b55865b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:16.756391 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:16.756361 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-metrics-certs\") pod \"maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:16.756551 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:16.756532 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf975\" (UniqueName: \"kubernetes.io/projected/22dcf276-7849-4247-89d2-98a53b55865b-kube-api-access-bf975\") pod \"maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:16.756850 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:16.756628 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:16.857388 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:16.857343 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:16.857571 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:16.857474 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22dcf276-7849-4247-89d2-98a53b55865b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:16.857571 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:39:16.857503 4859 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-serving-cert: secret "maas-api-e2e-worker-w0-90dc8d-serving-cert" not found Nov 04 05:39:16.857571 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:16.857519 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-metrics-certs\") pod \"maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:16.857571 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:39:16.857570 4859 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-maas-api-tls podName:22dcf276-7849-4247-89d2-98a53b55865b nodeName:}" failed. No retries permitted until 2026-11-04 05:39:17.357547973 +0000 UTC m=+727.359183676 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-maas-api-tls") pod "maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" (UID: "22dcf276-7849-4247-89d2-98a53b55865b") : secret "maas-api-e2e-worker-w0-90dc8d-serving-cert" not found Nov 04 05:39:16.857840 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:16.857631 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf975\" (UniqueName: \"kubernetes.io/projected/22dcf276-7849-4247-89d2-98a53b55865b-kube-api-access-bf975\") pod \"maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:16.858319 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:16.858295 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22dcf276-7849-4247-89d2-98a53b55865b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:16.859949 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:16.859932 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-metrics-certs\") pod \"maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:16.923143 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:16.923116 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf975\" (UniqueName: \"kubernetes.io/projected/22dcf276-7849-4247-89d2-98a53b55865b-kube-api-access-bf975\") pod \"maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:17.363161 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:17.363075 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:17.365489 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:17.365467 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:17.552684 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:17.552644 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:17.958550 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:17.957983 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr"] Nov 04 05:39:17.960769 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:39:17.960730 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22dcf276_7849_4247_89d2_98a53b55865b.slice/crio-b52bb6a370b04de01e5259fcd5e223985b240948663c3a6c4e4ba1cc2073dbda WatchSource:0}: Error finding container b52bb6a370b04de01e5259fcd5e223985b240948663c3a6c4e4ba1cc2073dbda: Status 404 returned error can't find the container with id b52bb6a370b04de01e5259fcd5e223985b240948663c3a6c4e4ba1cc2073dbda Nov 04 05:39:18.622027 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:18.621992 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" event={"ID":"22dcf276-7849-4247-89d2-98a53b55865b","Type":"ContainerStarted","Data":"b52bb6a370b04de01e5259fcd5e223985b240948663c3a6c4e4ba1cc2073dbda"} Nov 04 05:39:19.628763 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:19.628724 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" event={"ID":"22dcf276-7849-4247-89d2-98a53b55865b","Type":"ContainerStarted","Data":"93088e851cc6b7398069f10327690909bc7150673f27b5058e63f86ede4b8233"} Nov 04 05:39:19.629247 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:19.628955 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:19.760857 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:19.760793 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" podStartSLOduration=2.66719388 podStartE2EDuration="3.760776771s" totalImagesPullingTime="1.093582891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:39:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:39:19.75913201 +0000 UTC m=+729.760767735" watchObservedRunningTime="2026-11-04 05:39:19.760776771 +0000 UTC m=+729.762412496" Nov 04 05:39:20.565259 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:20.565232 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" Nov 04 05:39:20.565459 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:20.565298 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" Nov 04 05:39:22.592865 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:22.592835 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" Nov 04 05:39:25.376815 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:25.376770 4859 patch_prober.go:29] interesting pod/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 05:39:25.377190 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:25.376842 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" podUID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 05:39:25.641208 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:25.641141 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:39:26.673419 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:26.673394 4859 generic.go:356] "Generic (PLEG): container finished" podID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" containerID="a8d30f9bcd6b72a9f1b3fbce18e5a4e71d37e61b5e446510c32d858d4628876b" exitCode=0 Nov 04 05:39:26.673758 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:26.673471 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" event={"ID":"6fda8c73-74bb-49ce-b03e-e5517015ee6f","Type":"ContainerDied","Data":"a8d30f9bcd6b72a9f1b3fbce18e5a4e71d37e61b5e446510c32d858d4628876b"} Nov 04 05:39:26.673897 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:26.673882 4859 scope.go:122] "RemoveContainer" containerID="a8d30f9bcd6b72a9f1b3fbce18e5a4e71d37e61b5e446510c32d858d4628876b" Nov 04 05:39:26.675353 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:26.675321 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:39:26.675439 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:26.675393 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:39:26.675439 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:26.675425 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:39:27.719253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:27.719216 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" event={"ID":"6fda8c73-74bb-49ce-b03e-e5517015ee6f","Type":"ContainerStarted","Data":"ad34f79a1902e0a1110c8eb8d44901e0b3edb9e515922a90c2ca954205e214a8"} Nov 04 05:39:28.352542 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:39:28.352503 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:39:28.375947 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:28.375656 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:39:28.376085 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:28.375917 4859 patch_prober.go:29] interesting pod/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Nov 04 05:39:28.376085 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:28.376011 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" podUID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Nov 04 05:39:29.376333 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:29.376298 4859 patch_prober.go:29] interesting pod/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Nov 04 05:39:29.376789 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:29.376355 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" podUID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Nov 04 05:39:30.375996 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:30.375962 4859 patch_prober.go:29] interesting pod/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Nov 04 05:39:30.376186 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:30.376035 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" podUID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Nov 04 05:39:31.375939 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:31.375900 4859 patch_prober.go:29] interesting pod/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Nov 04 05:39:31.376362 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:31.375960 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" podUID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Nov 04 05:39:32.375978 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:32.375939 4859 patch_prober.go:29] interesting pod/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Nov 04 05:39:32.376351 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:32.376025 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" podUID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Nov 04 05:39:33.395689 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:33.389998 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:39:33.395689 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:33.391327 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:39:33.401885 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:33.401857 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:39:37.805319 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:37.805288 4859 generic.go:356] "Generic (PLEG): container finished" podID="c684e324-0d8e-438d-b589-8a671f7c8b18" containerID="513229f772e0aad21292d0ad06222275823360bb94f81f2952b71caea3516569" exitCode=0 Nov 04 05:39:37.805319 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:37.805321 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" event={"ID":"c684e324-0d8e-438d-b589-8a671f7c8b18","Type":"ContainerDied","Data":"513229f772e0aad21292d0ad06222275823360bb94f81f2952b71caea3516569"} Nov 04 05:39:37.805844 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:37.805693 4859 scope.go:122] "RemoveContainer" containerID="513229f772e0aad21292d0ad06222275823360bb94f81f2952b71caea3516569" Nov 04 05:39:37.806978 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:37.806948 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:39:37.807066 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:37.807016 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:39:37.807066 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:37.807047 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:39:38.815557 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:38.815514 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" event={"ID":"c684e324-0d8e-438d-b589-8a671f7c8b18","Type":"ContainerStarted","Data":"76825840171dd5bd48603ac811cedb4af5e01091aec7294fce5b8dfb05a76b4e"} Nov 04 05:39:39.044269 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:39.044235 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:39:39.045325 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:39.045294 4859 patch_prober.go:29] interesting pod/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Nov 04 05:39:39.045447 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:39.045344 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" podUID="c684e324-0d8e-438d-b589-8a671f7c8b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Nov 04 05:39:39.333640 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:39.333608 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:39:40.044094 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:40.044053 4859 patch_prober.go:29] interesting pod/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Nov 04 05:39:40.044490 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:40.044107 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" podUID="c684e324-0d8e-438d-b589-8a671f7c8b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Nov 04 05:39:41.044718 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:41.044658 4859 patch_prober.go:29] interesting pod/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Nov 04 05:39:41.045093 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:41.044738 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" podUID="c684e324-0d8e-438d-b589-8a671f7c8b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Nov 04 05:39:42.044171 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:42.044136 4859 patch_prober.go:29] interesting pod/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Nov 04 05:39:42.044346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:42.044186 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" podUID="c684e324-0d8e-438d-b589-8a671f7c8b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Nov 04 05:39:43.044438 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:43.044401 4859 patch_prober.go:29] interesting pod/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Nov 04 05:39:43.044849 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:43.044455 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" podUID="c684e324-0d8e-438d-b589-8a671f7c8b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Nov 04 05:39:44.048117 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:44.048086 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:39:44.049075 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:39:44.049054 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:39:58.342706 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:39:58.342617 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:40:28.341266 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:40:28.341233 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:40:38.738846 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.738805 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc"] Nov 04 05:40:38.742977 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.742955 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.788467 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.788423 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-home\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.788648 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.788484 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-tls-certs\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.788648 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.788599 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-model-cache\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.788750 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.788649 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-dshm\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.788894 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.788874 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4s2l\" (UniqueName: \"kubernetes.io/projected/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-kube-api-access-z4s2l\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.789008 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.788983 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-tmp-dir\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.854133 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.854097 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-283815\"/\"simulator-w5-283815-kserve-self-signed-certs\"" Nov 04 05:40:38.854133 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.854098 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-283815\"/\"kube-root-ca.crt\"" Nov 04 05:40:38.854428 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.854135 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-283815\"/\"default-dockercfg-jdn6x\"" Nov 04 05:40:38.854428 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.854194 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-283815\"/\"openshift-service-ca.crt\"" Nov 04 05:40:38.856717 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.856694 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc"] Nov 04 05:40:38.907127 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.898551 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4s2l\" (UniqueName: \"kubernetes.io/projected/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-kube-api-access-z4s2l\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.910626 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.907577 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-tmp-dir\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.910626 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.908968 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-home\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.910626 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.909019 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-tls-certs\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.910626 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.909060 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-model-cache\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.910626 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.909096 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-dshm\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.910626 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.909832 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-home\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.910626 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.908105 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-tmp-dir\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.910626 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.910576 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-model-cache\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.915643 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.915582 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-dshm\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.917349 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.917314 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-tls-certs\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:38.968588 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:38.968557 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4s2l\" (UniqueName: \"kubernetes.io/projected/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-kube-api-access-z4s2l\") pod \"simulator-w5-283815-kserve-549cccb89-rrrnc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:39.065157 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:39.065055 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:39.458948 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:39.458913 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc"] Nov 04 05:40:39.462228 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:40:39.462197 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb34a4fed_23f4_456d_86a7_828eaf8c8bbc.slice/crio-c58cfca1058c60c6ede38818457b968bd50de66addbe5c4e9c050c55eba6bf38 WatchSource:0}: Error finding container c58cfca1058c60c6ede38818457b968bd50de66addbe5c4e9c050c55eba6bf38: Status 404 returned error can't find the container with id c58cfca1058c60c6ede38818457b968bd50de66addbe5c4e9c050c55eba6bf38 Nov 04 05:40:40.182401 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:40.182362 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" event={"ID":"b34a4fed-23f4-456d-86a7-828eaf8c8bbc","Type":"ContainerStarted","Data":"6de8ebc46daed2d9c1b7dd0a3b2eea2c15d3203e3aaf7e141485dd118309429b"} Nov 04 05:40:40.182401 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:40.182403 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" event={"ID":"b34a4fed-23f4-456d-86a7-828eaf8c8bbc","Type":"ContainerStarted","Data":"c58cfca1058c60c6ede38818457b968bd50de66addbe5c4e9c050c55eba6bf38"} Nov 04 05:40:40.314535 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:40.314483 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" podStartSLOduration=2.314467753 podStartE2EDuration="2.314467753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:40:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:40:40.312463238 +0000 UTC m=+810.314098964" watchObservedRunningTime="2026-11-04 05:40:40.314467753 +0000 UTC m=+810.316103478" Nov 04 05:40:41.065558 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:41.065515 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:49.065383 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:49.065351 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:49.078397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:49.078368 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:49.253892 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:40:49.253856 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:40:58.341349 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:40:58.341313 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:41:28.343251 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:41:28.343207 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:41:37.724821 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.724791 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv"] Nov 04 05:41:37.728754 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.728735 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.785812 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.785779 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-a84d48-openshift-default-dockercfg-cqcsm\"" Nov 04 05:41:37.796366 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.796329 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-envoy\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.796366 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.796369 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnlcv\" (UniqueName: \"kubernetes.io/projected/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-kube-api-access-gnlcv\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.796588 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.796484 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-data\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.796588 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.796522 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istiod-ca-cert\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.796698 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.796582 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-workload-certs\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.796698 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.796620 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-podinfo\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.796698 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.796641 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-token\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.796867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.796780 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-workload-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.796867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.796835 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-credential-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.843993 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.843953 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv"] Nov 04 05:41:37.898208 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.898174 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-token\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.898208 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.898214 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-workload-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.898416 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.898233 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-credential-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.898416 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.898298 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-envoy\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.898416 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.898319 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnlcv\" (UniqueName: \"kubernetes.io/projected/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-kube-api-access-gnlcv\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.898416 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.898390 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-data\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.898622 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.898449 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istiod-ca-cert\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.898622 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.898502 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-workload-certs\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.898765 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.898657 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-workload-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.898765 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.898715 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-credential-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.898842 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.898805 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-data\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.898955 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.898929 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-workload-certs\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.899204 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.899180 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-podinfo\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.899337 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.899269 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istiod-ca-cert\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.900897 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.900874 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-envoy\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.901422 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.901406 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-podinfo\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.965205 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.965130 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnlcv\" (UniqueName: \"kubernetes.io/projected/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-kube-api-access-gnlcv\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:37.965386 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:37.965242 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-token\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:38.042452 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:38.042366 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:38.494741 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:38.494713 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv"] Nov 04 05:41:38.497765 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:41:38.497739 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e75fb43_a2fa_48d1_bc71_8f4f3514b817.slice/crio-4b6f65268ef355d36fe55e81e6d3a0a028ea9792b6d57fce988b272c36c11f6d WatchSource:0}: Error finding container 4b6f65268ef355d36fe55e81e6d3a0a028ea9792b6d57fce988b272c36c11f6d: Status 404 returned error can't find the container with id 4b6f65268ef355d36fe55e81e6d3a0a028ea9792b6d57fce988b272c36c11f6d Nov 04 05:41:38.500124 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:38.500094 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:41:38.500211 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:38.500161 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:41:38.500211 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:38.500192 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:41:38.551646 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:38.551613 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" event={"ID":"0e75fb43-a2fa-48d1-bc71-8f4f3514b817","Type":"ContainerStarted","Data":"4b6f65268ef355d36fe55e81e6d3a0a028ea9792b6d57fce988b272c36c11f6d"} Nov 04 05:41:39.559329 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:39.559288 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" event={"ID":"0e75fb43-a2fa-48d1-bc71-8f4f3514b817","Type":"ContainerStarted","Data":"93ea2b28bd9cc20f3d9efd61b71b43c1e95c84958a63c1299fc995657b81060e"} Nov 04 05:41:39.743345 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:39.743290 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" podStartSLOduration=2.74327728 podStartE2EDuration="2.74327728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:41:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:41:39.742279301 +0000 UTC m=+869.743915026" watchObservedRunningTime="2026-11-04 05:41:39.74327728 +0000 UTC m=+869.744913005" Nov 04 05:41:40.043074 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:40.043037 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:40.044824 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:40.044801 4859 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Nov 04 05:41:40.044938 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:40.044846 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" podUID="0e75fb43-a2fa-48d1-bc71-8f4f3514b817" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Nov 04 05:41:41.043295 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:41.043260 4859 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Nov 04 05:41:41.043727 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:41.043313 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" podUID="0e75fb43-a2fa-48d1-bc71-8f4f3514b817" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Nov 04 05:41:42.043635 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:42.043591 4859 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Nov 04 05:41:42.044077 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:42.043688 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" podUID="0e75fb43-a2fa-48d1-bc71-8f4f3514b817" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Nov 04 05:41:43.042852 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:43.042815 4859 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Nov 04 05:41:43.043090 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:43.042873 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" podUID="0e75fb43-a2fa-48d1-bc71-8f4f3514b817" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Nov 04 05:41:44.043266 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:44.043233 4859 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Nov 04 05:41:44.043687 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:44.043298 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" podUID="0e75fb43-a2fa-48d1-bc71-8f4f3514b817" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Nov 04 05:41:45.042858 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:45.042821 4859 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Nov 04 05:41:45.043031 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:45.042888 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" podUID="0e75fb43-a2fa-48d1-bc71-8f4f3514b817" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Nov 04 05:41:46.541637 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:46.541609 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:46.542175 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:46.542008 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:46.542959 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:41:46.542926 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:41:58.377318 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:41:58.377230 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:42:10.197828 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:10.197804 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/2.log" Nov 04 05:42:10.201811 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:10.201785 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/2.log" Nov 04 05:42:28.342412 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:42:28.342376 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:42:41.654416 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.654375 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6"] Nov 04 05:42:41.658895 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.658870 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.715856 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.715832 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-fbbad9\"/\"openshift-service-ca.crt\"" Nov 04 05:42:41.777658 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.777619 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6"] Nov 04 05:42:41.780725 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.777976 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-fbbad9\"/\"kube-root-ca.crt\"" Nov 04 05:42:41.781096 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.778632 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-fbbad9\"/\"default-dockercfg-prgbv\"" Nov 04 05:42:41.781338 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.780581 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-fbbad9\"/\"premium-w6-fbbad9-kserve-self-signed-certs\"" Nov 04 05:42:41.819500 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.819472 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-dshm\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.819688 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.819515 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wn2h\" (UniqueName: \"kubernetes.io/projected/824e4ea9-5112-4827-9392-9b22b87e3a58-kube-api-access-8wn2h\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.819688 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.819538 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-model-cache\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.819773 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.819721 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-home\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.819834 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.819814 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-tmp-dir\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.819888 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.819876 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/824e4ea9-5112-4827-9392-9b22b87e3a58-tls-certs\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.920716 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.920690 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-tmp-dir\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.920907 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.920743 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/824e4ea9-5112-4827-9392-9b22b87e3a58-tls-certs\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.920907 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.920794 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-dshm\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.920907 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.920822 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wn2h\" (UniqueName: \"kubernetes.io/projected/824e4ea9-5112-4827-9392-9b22b87e3a58-kube-api-access-8wn2h\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.920907 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.920853 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-model-cache\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.921138 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.920908 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-home\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.921200 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.921168 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-tmp-dir\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.921259 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.921240 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-model-cache\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.921366 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.921348 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-home\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.923121 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.923102 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-dshm\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.923288 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.923272 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/824e4ea9-5112-4827-9392-9b22b87e3a58-tls-certs\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:41.987471 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:41.987439 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wn2h\" (UniqueName: \"kubernetes.io/projected/824e4ea9-5112-4827-9392-9b22b87e3a58-kube-api-access-8wn2h\") pod \"premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:42.272855 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:42.272765 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:42.666529 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:42.666499 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6"] Nov 04 05:42:42.670015 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:42:42.669991 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod824e4ea9_5112_4827_9392_9b22b87e3a58.slice/crio-509716d5e177229b85eb55645787115c708a6d3a59f537ede30be96b9bd8aec9 WatchSource:0}: Error finding container 509716d5e177229b85eb55645787115c708a6d3a59f537ede30be96b9bd8aec9: Status 404 returned error can't find the container with id 509716d5e177229b85eb55645787115c708a6d3a59f537ede30be96b9bd8aec9 Nov 04 05:42:42.965608 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:42.965528 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" event={"ID":"824e4ea9-5112-4827-9392-9b22b87e3a58","Type":"ContainerStarted","Data":"7f607cbbaa0de218cb5049118926da6f5b301cc140dba75869c09e3e9074aa08"} Nov 04 05:42:42.965608 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:42.965568 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" event={"ID":"824e4ea9-5112-4827-9392-9b22b87e3a58","Type":"ContainerStarted","Data":"509716d5e177229b85eb55645787115c708a6d3a59f537ede30be96b9bd8aec9"} Nov 04 05:42:43.092738 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:43.092681 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" podStartSLOduration=2.092644408 podStartE2EDuration="2.092644408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:42:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:42:43.090815916 +0000 UTC m=+933.092451649" watchObservedRunningTime="2026-11-04 05:42:43.092644408 +0000 UTC m=+933.094280135" Nov 04 05:42:43.272953 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:43.272874 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:51.064857 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:51.063487 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524"] Nov 04 05:42:51.064857 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:51.063830 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" podUID="1d8417ad-b883-4d9e-9b80-d352d45202c4" containerName="istio-proxy" containerID="cri-o://180db0947f31576d3bbf23eab43a9ad3492fb8d6c0b9d9213905249128f90391" gracePeriod=30 Nov 04 05:42:51.112392 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:51.112354 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz"] Nov 04 05:42:51.112620 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:51.112596 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" podUID="52754caf-6be9-4c0f-a8f4-52c2aedf159e" containerName="istio-proxy" containerID="cri-o://3dbc0f5cb04d19d742e69a7f9e7bd8f43d04addd4ae9a7ba344b8344d12e5377" gracePeriod=30 Nov 04 05:42:51.114747 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:51.114719 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq"] Nov 04 05:42:51.114919 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:51.114898 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" podUID="57a4e230-bf07-4f13-9175-b3589b86ecca" containerName="istio-proxy" containerID="cri-o://af7ac12f2352f996525146febbbf4ac47620024f73789590d4010ce03c1ba356" gracePeriod=30 Nov 04 05:42:52.273739 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:52.273707 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:52.286511 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:52.286489 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:53.033824 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:53.033797 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:42:56.412539 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.412510 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:42:56.439944 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.439924 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:42:56.459025 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.459006 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:42:56.474560 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474539 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-data\") pod \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " Nov 04 05:42:56.474681 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474575 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-workload-socket\") pod \"57a4e230-bf07-4f13-9175-b3589b86ecca\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " Nov 04 05:42:56.474681 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474598 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-podinfo\") pod \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " Nov 04 05:42:56.474681 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474620 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-credential-socket\") pod \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " Nov 04 05:42:56.474681 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474650 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52754caf-6be9-4c0f-a8f4-52c2aedf159e-kube-api-access-2pkqq\" (UniqueName: \"kubernetes.io/projected/52754caf-6be9-4c0f-a8f4-52c2aedf159e-kube-api-access-2pkqq\") pod \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " Nov 04 05:42:56.474887 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474696 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-envoy\") pod \"57a4e230-bf07-4f13-9175-b3589b86ecca\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " Nov 04 05:42:56.474887 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474728 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-workload-certs\") pod \"57a4e230-bf07-4f13-9175-b3589b86ecca\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " Nov 04 05:42:56.474887 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474764 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57a4e230-bf07-4f13-9175-b3589b86ecca-kube-api-access-l597f\" (UniqueName: \"kubernetes.io/projected/57a4e230-bf07-4f13-9175-b3589b86ecca-kube-api-access-l597f\") pod \"57a4e230-bf07-4f13-9175-b3589b86ecca\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " Nov 04 05:42:56.474887 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474809 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-envoy\") pod \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " Nov 04 05:42:56.474887 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474843 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-workload-certs\") pod \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " Nov 04 05:42:56.474887 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474868 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57a4e230-bf07-4f13-9175-b3589b86ecca-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57a4e230-bf07-4f13-9175-b3589b86ecca-istiod-ca-cert\") pod \"57a4e230-bf07-4f13-9175-b3589b86ecca\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " Nov 04 05:42:56.474887 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474867 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-data" pod "52754caf-6be9-4c0f-a8f4-52c2aedf159e" (UID: "52754caf-6be9-4c0f-a8f4-52c2aedf159e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:42:56.474887 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474877 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-workload-socket" pod "57a4e230-bf07-4f13-9175-b3589b86ecca" (UID: "57a4e230-bf07-4f13-9175-b3589b86ecca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:42:56.475286 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474901 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-token\" (UniqueName: \"kubernetes.io/projected/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-token\") pod \"57a4e230-bf07-4f13-9175-b3589b86ecca\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " Nov 04 05:42:56.475286 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474917 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-data\") pod \"57a4e230-bf07-4f13-9175-b3589b86ecca\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " Nov 04 05:42:56.475286 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474944 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-workload-socket\") pod \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " Nov 04 05:42:56.475286 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474972 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-token\" (UniqueName: \"kubernetes.io/projected/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-token\") pod \"1d8417ad-b883-4d9e-9b80-d352d45202c4\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " Nov 04 05:42:56.475286 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.474989 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-data\") pod \"1d8417ad-b883-4d9e-9b80-d352d45202c4\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " Nov 04 05:42:56.475286 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475012 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-workload-socket\") pod \"1d8417ad-b883-4d9e-9b80-d352d45202c4\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " Nov 04 05:42:56.475286 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475034 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-credential-socket\") pod \"57a4e230-bf07-4f13-9175-b3589b86ecca\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " Nov 04 05:42:56.475286 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475033 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-workload-certs" pod "57a4e230-bf07-4f13-9175-b3589b86ecca" (UID: "57a4e230-bf07-4f13-9175-b3589b86ecca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:42:56.475286 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475138 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-podinfo\") pod \"1d8417ad-b883-4d9e-9b80-d352d45202c4\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " Nov 04 05:42:56.475286 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475161 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-credential-socket" pod "52754caf-6be9-4c0f-a8f4-52c2aedf159e" (UID: "52754caf-6be9-4c0f-a8f4-52c2aedf159e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:42:56.475286 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475205 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-podinfo\") pod \"57a4e230-bf07-4f13-9175-b3589b86ecca\" (UID: \"57a4e230-bf07-4f13-9175-b3589b86ecca\") " Nov 04 05:42:56.475286 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475245 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-workload-certs\") pod \"1d8417ad-b883-4d9e-9b80-d352d45202c4\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " Nov 04 05:42:56.475286 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475248 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-credential-socket" pod "57a4e230-bf07-4f13-9175-b3589b86ecca" (UID: "57a4e230-bf07-4f13-9175-b3589b86ecca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:42:56.475286 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475289 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1d8417ad-b883-4d9e-9b80-d352d45202c4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d8417ad-b883-4d9e-9b80-d352d45202c4-istiod-ca-cert\") pod \"1d8417ad-b883-4d9e-9b80-d352d45202c4\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " Nov 04 05:42:56.475978 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475349 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-credential-socket\") pod \"1d8417ad-b883-4d9e-9b80-d352d45202c4\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " Nov 04 05:42:56.475978 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475353 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-data" pod "57a4e230-bf07-4f13-9175-b3589b86ecca" (UID: "57a4e230-bf07-4f13-9175-b3589b86ecca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:42:56.475978 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475390 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-token\" (UniqueName: \"kubernetes.io/projected/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-token\") pod \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " Nov 04 05:42:56.475978 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475466 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istiod-ca-cert\") pod \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\" (UID: \"52754caf-6be9-4c0f-a8f4-52c2aedf159e\") " Nov 04 05:42:56.475978 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475467 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-workload-socket" pod "52754caf-6be9-4c0f-a8f4-52c2aedf159e" (UID: "52754caf-6be9-4c0f-a8f4-52c2aedf159e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:42:56.475978 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475498 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-envoy\") pod \"1d8417ad-b883-4d9e-9b80-d352d45202c4\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " Nov 04 05:42:56.475978 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475538 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-workload-certs" pod "52754caf-6be9-4c0f-a8f4-52c2aedf159e" (UID: "52754caf-6be9-4c0f-a8f4-52c2aedf159e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:42:56.475978 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475554 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d8417ad-b883-4d9e-9b80-d352d45202c4-kube-api-access-99lbh\" (UniqueName: \"kubernetes.io/projected/1d8417ad-b883-4d9e-9b80-d352d45202c4-kube-api-access-99lbh\") pod \"1d8417ad-b883-4d9e-9b80-d352d45202c4\" (UID: \"1d8417ad-b883-4d9e-9b80-d352d45202c4\") " Nov 04 05:42:56.475978 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.475848 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57a4e230-bf07-4f13-9175-b3589b86ecca-istiod-ca-cert" pod "57a4e230-bf07-4f13-9175-b3589b86ecca" (UID: "57a4e230-bf07-4f13-9175-b3589b86ecca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:42:56.476468 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.476117 4859 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-data\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.476468 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.476140 4859 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-workload-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.476468 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.476156 4859 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-credential-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.476468 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.476171 4859 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-workload-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.476468 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.476185 4859 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-workload-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.476468 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.476202 4859 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57a4e230-bf07-4f13-9175-b3589b86ecca-istiod-ca-cert\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.476468 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.476227 4859 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-data\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.476468 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.476242 4859 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-workload-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.476468 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.476259 4859 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-credential-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.476963 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.476842 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-data" pod "1d8417ad-b883-4d9e-9b80-d352d45202c4" (UID: "1d8417ad-b883-4d9e-9b80-d352d45202c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:42:56.476963 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.476923 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d8417ad-b883-4d9e-9b80-d352d45202c4-istiod-ca-cert" pod "1d8417ad-b883-4d9e-9b80-d352d45202c4" (UID: "1d8417ad-b883-4d9e-9b80-d352d45202c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:42:56.477590 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.477292 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-credential-socket" pod "1d8417ad-b883-4d9e-9b80-d352d45202c4" (UID: "1d8417ad-b883-4d9e-9b80-d352d45202c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:42:56.477590 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.477505 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-envoy" pod "52754caf-6be9-4c0f-a8f4-52c2aedf159e" (UID: "52754caf-6be9-4c0f-a8f4-52c2aedf159e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:42:56.477733 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.477605 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57a4e230-bf07-4f13-9175-b3589b86ecca-kube-api-access-l597f" pod "57a4e230-bf07-4f13-9175-b3589b86ecca" (UID: "57a4e230-bf07-4f13-9175-b3589b86ecca"). InnerVolumeSpecName "kube-api-access-l597f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:42:56.477797 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.477731 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istiod-ca-cert" pod "52754caf-6be9-4c0f-a8f4-52c2aedf159e" (UID: "52754caf-6be9-4c0f-a8f4-52c2aedf159e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:42:56.477861 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.477842 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-workload-certs" pod "1d8417ad-b883-4d9e-9b80-d352d45202c4" (UID: "1d8417ad-b883-4d9e-9b80-d352d45202c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:42:56.478099 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.478019 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-workload-socket" pod "1d8417ad-b883-4d9e-9b80-d352d45202c4" (UID: "1d8417ad-b883-4d9e-9b80-d352d45202c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:42:56.478476 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.478442 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-token" pod "1d8417ad-b883-4d9e-9b80-d352d45202c4" (UID: "1d8417ad-b883-4d9e-9b80-d352d45202c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:42:56.478568 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.478483 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52754caf-6be9-4c0f-a8f4-52c2aedf159e-kube-api-access-2pkqq" pod "52754caf-6be9-4c0f-a8f4-52c2aedf159e" (UID: "52754caf-6be9-4c0f-a8f4-52c2aedf159e"). InnerVolumeSpecName "kube-api-access-2pkqq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:42:56.479042 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.479019 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-podinfo" pod "52754caf-6be9-4c0f-a8f4-52c2aedf159e" (UID: "52754caf-6be9-4c0f-a8f4-52c2aedf159e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:42:56.479406 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.479332 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d8417ad-b883-4d9e-9b80-d352d45202c4-kube-api-access-99lbh" pod "1d8417ad-b883-4d9e-9b80-d352d45202c4" (UID: "1d8417ad-b883-4d9e-9b80-d352d45202c4"). InnerVolumeSpecName "kube-api-access-99lbh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:42:56.479406 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.479387 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-envoy" pod "57a4e230-bf07-4f13-9175-b3589b86ecca" (UID: "57a4e230-bf07-4f13-9175-b3589b86ecca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:42:56.479582 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.479560 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-token" pod "57a4e230-bf07-4f13-9175-b3589b86ecca" (UID: "57a4e230-bf07-4f13-9175-b3589b86ecca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:42:56.479885 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.479862 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-podinfo" pod "1d8417ad-b883-4d9e-9b80-d352d45202c4" (UID: "1d8417ad-b883-4d9e-9b80-d352d45202c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:42:56.479940 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.479879 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-podinfo" pod "57a4e230-bf07-4f13-9175-b3589b86ecca" (UID: "57a4e230-bf07-4f13-9175-b3589b86ecca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:42:56.480199 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.480186 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-envoy" pod "1d8417ad-b883-4d9e-9b80-d352d45202c4" (UID: "1d8417ad-b883-4d9e-9b80-d352d45202c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:42:56.480566 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.480552 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-token" pod "52754caf-6be9-4c0f-a8f4-52c2aedf159e" (UID: "52754caf-6be9-4c0f-a8f4-52c2aedf159e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:42:56.577678 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577643 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l597f\" (UniqueName: \"kubernetes.io/projected/57a4e230-bf07-4f13-9175-b3589b86ecca-kube-api-access-l597f\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.577798 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577695 4859 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-envoy\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.577798 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577706 4859 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.577798 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577715 4859 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.577798 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577723 4859 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-data\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.577798 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577731 4859 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-workload-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.577798 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577740 4859 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-podinfo\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.577798 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577748 4859 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-podinfo\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.577798 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577759 4859 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-workload-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.577798 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577771 4859 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d8417ad-b883-4d9e-9b80-d352d45202c4-istiod-ca-cert\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.577798 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577779 4859 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-credential-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.577798 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577788 4859 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.577798 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577798 4859 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istiod-ca-cert\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.578156 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577806 4859 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d8417ad-b883-4d9e-9b80-d352d45202c4-istio-envoy\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.578156 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577816 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99lbh\" (UniqueName: \"kubernetes.io/projected/1d8417ad-b883-4d9e-9b80-d352d45202c4-kube-api-access-99lbh\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.578156 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577825 4859 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52754caf-6be9-4c0f-a8f4-52c2aedf159e-istio-podinfo\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.578156 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577834 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pkqq\" (UniqueName: \"kubernetes.io/projected/52754caf-6be9-4c0f-a8f4-52c2aedf159e-kube-api-access-2pkqq\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:56.578156 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:56.577848 4859 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57a4e230-bf07-4f13-9175-b3589b86ecca-istio-envoy\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:42:57.049120 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.049082 4859 generic.go:356] "Generic (PLEG): container finished" podID="1d8417ad-b883-4d9e-9b80-d352d45202c4" containerID="180db0947f31576d3bbf23eab43a9ad3492fb8d6c0b9d9213905249128f90391" exitCode=0 Nov 04 05:42:57.049297 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.049133 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" Nov 04 05:42:57.049297 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.049169 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" event={"ID":"1d8417ad-b883-4d9e-9b80-d352d45202c4","Type":"ContainerDied","Data":"180db0947f31576d3bbf23eab43a9ad3492fb8d6c0b9d9213905249128f90391"} Nov 04 05:42:57.049297 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.049228 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524" event={"ID":"1d8417ad-b883-4d9e-9b80-d352d45202c4","Type":"ContainerDied","Data":"0e5c2d1934dd0ccd837210d95aebc49a99804eaf86c4c494f9f858dec8ec5c29"} Nov 04 05:42:57.049297 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.049254 4859 scope.go:122] "RemoveContainer" containerID="180db0947f31576d3bbf23eab43a9ad3492fb8d6c0b9d9213905249128f90391" Nov 04 05:42:57.050870 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.050843 4859 generic.go:356] "Generic (PLEG): container finished" podID="57a4e230-bf07-4f13-9175-b3589b86ecca" containerID="af7ac12f2352f996525146febbbf4ac47620024f73789590d4010ce03c1ba356" exitCode=0 Nov 04 05:42:57.050985 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.050869 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" event={"ID":"57a4e230-bf07-4f13-9175-b3589b86ecca","Type":"ContainerDied","Data":"af7ac12f2352f996525146febbbf4ac47620024f73789590d4010ce03c1ba356"} Nov 04 05:42:57.050985 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.050905 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" Nov 04 05:42:57.050985 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.050912 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq" event={"ID":"57a4e230-bf07-4f13-9175-b3589b86ecca","Type":"ContainerDied","Data":"1fc23f91387d37d3cb1f1bb93859045c30173e303707c59d0ff18d4981d345d6"} Nov 04 05:42:57.052361 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.052334 4859 generic.go:356] "Generic (PLEG): container finished" podID="52754caf-6be9-4c0f-a8f4-52c2aedf159e" containerID="3dbc0f5cb04d19d742e69a7f9e7bd8f43d04addd4ae9a7ba344b8344d12e5377" exitCode=0 Nov 04 05:42:57.052475 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.052365 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" event={"ID":"52754caf-6be9-4c0f-a8f4-52c2aedf159e","Type":"ContainerDied","Data":"3dbc0f5cb04d19d742e69a7f9e7bd8f43d04addd4ae9a7ba344b8344d12e5377"} Nov 04 05:42:57.052475 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.052389 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" Nov 04 05:42:57.052475 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.052403 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz" event={"ID":"52754caf-6be9-4c0f-a8f4-52c2aedf159e","Type":"ContainerDied","Data":"1055e520f1780cd163fb5bd3dcfc3c91de37be21a4033aa3faa1a5fc1bc2f997"} Nov 04 05:42:57.061598 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.061575 4859 scope.go:122] "RemoveContainer" containerID="180db0947f31576d3bbf23eab43a9ad3492fb8d6c0b9d9213905249128f90391" Nov 04 05:42:57.061901 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:42:57.061882 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"180db0947f31576d3bbf23eab43a9ad3492fb8d6c0b9d9213905249128f90391\": container with ID starting with 180db0947f31576d3bbf23eab43a9ad3492fb8d6c0b9d9213905249128f90391 not found: ID does not exist" containerID="180db0947f31576d3bbf23eab43a9ad3492fb8d6c0b9d9213905249128f90391" Nov 04 05:42:57.062003 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.061906 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"180db0947f31576d3bbf23eab43a9ad3492fb8d6c0b9d9213905249128f90391"} err="failed to get container status \"180db0947f31576d3bbf23eab43a9ad3492fb8d6c0b9d9213905249128f90391\": rpc error: code = NotFound desc = could not find container \"180db0947f31576d3bbf23eab43a9ad3492fb8d6c0b9d9213905249128f90391\": container with ID starting with 180db0947f31576d3bbf23eab43a9ad3492fb8d6c0b9d9213905249128f90391 not found: ID does not exist" Nov 04 05:42:57.062003 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.061925 4859 scope.go:122] "RemoveContainer" containerID="af7ac12f2352f996525146febbbf4ac47620024f73789590d4010ce03c1ba356" Nov 04 05:42:57.070910 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.070895 4859 scope.go:122] "RemoveContainer" containerID="af7ac12f2352f996525146febbbf4ac47620024f73789590d4010ce03c1ba356" Nov 04 05:42:57.071184 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:42:57.071165 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af7ac12f2352f996525146febbbf4ac47620024f73789590d4010ce03c1ba356\": container with ID starting with af7ac12f2352f996525146febbbf4ac47620024f73789590d4010ce03c1ba356 not found: ID does not exist" containerID="af7ac12f2352f996525146febbbf4ac47620024f73789590d4010ce03c1ba356" Nov 04 05:42:57.071220 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.071190 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af7ac12f2352f996525146febbbf4ac47620024f73789590d4010ce03c1ba356"} err="failed to get container status \"af7ac12f2352f996525146febbbf4ac47620024f73789590d4010ce03c1ba356\": rpc error: code = NotFound desc = could not find container \"af7ac12f2352f996525146febbbf4ac47620024f73789590d4010ce03c1ba356\": container with ID starting with af7ac12f2352f996525146febbbf4ac47620024f73789590d4010ce03c1ba356 not found: ID does not exist" Nov 04 05:42:57.071220 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.071206 4859 scope.go:122] "RemoveContainer" containerID="3dbc0f5cb04d19d742e69a7f9e7bd8f43d04addd4ae9a7ba344b8344d12e5377" Nov 04 05:42:57.082546 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.082514 4859 scope.go:122] "RemoveContainer" containerID="3dbc0f5cb04d19d742e69a7f9e7bd8f43d04addd4ae9a7ba344b8344d12e5377" Nov 04 05:42:57.082957 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:42:57.082900 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dbc0f5cb04d19d742e69a7f9e7bd8f43d04addd4ae9a7ba344b8344d12e5377\": container with ID starting with 3dbc0f5cb04d19d742e69a7f9e7bd8f43d04addd4ae9a7ba344b8344d12e5377 not found: ID does not exist" containerID="3dbc0f5cb04d19d742e69a7f9e7bd8f43d04addd4ae9a7ba344b8344d12e5377" Nov 04 05:42:57.082957 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.082933 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dbc0f5cb04d19d742e69a7f9e7bd8f43d04addd4ae9a7ba344b8344d12e5377"} err="failed to get container status \"3dbc0f5cb04d19d742e69a7f9e7bd8f43d04addd4ae9a7ba344b8344d12e5377\": rpc error: code = NotFound desc = could not find container \"3dbc0f5cb04d19d742e69a7f9e7bd8f43d04addd4ae9a7ba344b8344d12e5377\": container with ID starting with 3dbc0f5cb04d19d742e69a7f9e7bd8f43d04addd4ae9a7ba344b8344d12e5377 not found: ID does not exist" Nov 04 05:42:57.227330 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.227302 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz"] Nov 04 05:42:57.232220 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.232191 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jrfvz"] Nov 04 05:42:57.350722 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.350646 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq"] Nov 04 05:42:57.354655 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.354630 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5s5sq"] Nov 04 05:42:57.474099 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.474071 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524"] Nov 04 05:42:57.478590 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:57.478567 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s2524"] Nov 04 05:42:58.115173 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:58.115143 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d8417ad-b883-4d9e-9b80-d352d45202c4" path="/var/lib/kubelet/pods/1d8417ad-b883-4d9e-9b80-d352d45202c4/volumes" Nov 04 05:42:58.115569 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:58.115556 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52754caf-6be9-4c0f-a8f4-52c2aedf159e" path="/var/lib/kubelet/pods/52754caf-6be9-4c0f-a8f4-52c2aedf159e/volumes" Nov 04 05:42:58.115978 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:42:58.115966 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57a4e230-bf07-4f13-9175-b3589b86ecca" path="/var/lib/kubelet/pods/57a4e230-bf07-4f13-9175-b3589b86ecca/volumes" Nov 04 05:42:58.340428 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:42:58.340393 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:43:10.575342 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.575270 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s"] Nov 04 05:43:10.575872 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.575857 4859 cpu_manager.go:414] "containerMap: removing container" podUID="57a4e230-bf07-4f13-9175-b3589b86ecca" containerName="istio-proxy" Nov 04 05:43:10.575921 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.575874 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57a4e230-bf07-4f13-9175-b3589b86ecca" containerName="istio-proxy" Nov 04 05:43:10.575921 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.575891 4859 cpu_manager.go:414] "containerMap: removing container" podUID="1d8417ad-b883-4d9e-9b80-d352d45202c4" containerName="istio-proxy" Nov 04 05:43:10.575921 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.575896 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d8417ad-b883-4d9e-9b80-d352d45202c4" containerName="istio-proxy" Nov 04 05:43:10.576013 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.575922 4859 cpu_manager.go:414] "containerMap: removing container" podUID="52754caf-6be9-4c0f-a8f4-52c2aedf159e" containerName="istio-proxy" Nov 04 05:43:10.576013 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.575927 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52754caf-6be9-4c0f-a8f4-52c2aedf159e" containerName="istio-proxy" Nov 04 05:43:10.576013 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.576011 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="57a4e230-bf07-4f13-9175-b3589b86ecca" containerName="istio-proxy" Nov 04 05:43:10.576102 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.576020 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d8417ad-b883-4d9e-9b80-d352d45202c4" containerName="istio-proxy" Nov 04 05:43:10.576102 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.576030 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="52754caf-6be9-4c0f-a8f4-52c2aedf159e" containerName="istio-proxy" Nov 04 05:43:10.581240 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.581224 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.638287 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.638258 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-961485-openshift-default-dockercfg-qhhl7\"" Nov 04 05:43:10.695015 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.694985 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s"] Nov 04 05:43:10.728316 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.728285 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.728484 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.728348 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.728484 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.728429 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.728610 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.728508 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkjg7\" (UniqueName: \"kubernetes.io/projected/7e5b156e-c723-400a-b76e-4d914ad58712-kube-api-access-jkjg7\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.728727 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.728707 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.728812 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.728744 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e5b156e-c723-400a-b76e-4d914ad58712-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.728812 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.728774 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.728924 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.728879 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e5b156e-c723-400a-b76e-4d914ad58712-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.728984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.728927 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e5b156e-c723-400a-b76e-4d914ad58712-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.830464 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.830389 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.830464 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.830424 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e5b156e-c723-400a-b76e-4d914ad58712-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.830464 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.830444 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.830736 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.830471 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e5b156e-c723-400a-b76e-4d914ad58712-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.830736 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.830507 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e5b156e-c723-400a-b76e-4d914ad58712-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.830736 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.830543 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.830903 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.830731 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.830903 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.830777 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.830903 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.830808 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkjg7\" (UniqueName: \"kubernetes.io/projected/7e5b156e-c723-400a-b76e-4d914ad58712-kube-api-access-jkjg7\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.830903 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.830881 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.831091 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.830927 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.831091 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.830959 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.831232 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.831211 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.831318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.831303 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e5b156e-c723-400a-b76e-4d914ad58712-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.833193 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.833169 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.833348 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.833328 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e5b156e-c723-400a-b76e-4d914ad58712-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.903290 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.903266 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkjg7\" (UniqueName: \"kubernetes.io/projected/7e5b156e-c723-400a-b76e-4d914ad58712-kube-api-access-jkjg7\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:10.910659 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:10.910638 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e5b156e-c723-400a-b76e-4d914ad58712-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:11.195087 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:11.195063 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:11.650383 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:11.650302 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s"] Nov 04 05:43:11.653622 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:43:11.653594 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e5b156e_c723_400a_b76e_4d914ad58712.slice/crio-e34411db6f73291ed53068ec6770a96ba2effb9046e0e8bb714c57b2c6703391 WatchSource:0}: Error finding container e34411db6f73291ed53068ec6770a96ba2effb9046e0e8bb714c57b2c6703391: Status 404 returned error can't find the container with id e34411db6f73291ed53068ec6770a96ba2effb9046e0e8bb714c57b2c6703391 Nov 04 05:43:11.656065 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:11.656029 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:43:11.656146 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:11.656113 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:43:11.656146 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:11.656143 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:43:12.139515 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:12.139480 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" event={"ID":"7e5b156e-c723-400a-b76e-4d914ad58712","Type":"ContainerStarted","Data":"dc42e57ca337d7f0b9ca62ec7eb62f3114d35e3db6988407e20ac7db4f1e8760"} Nov 04 05:43:12.139714 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:12.139536 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" event={"ID":"7e5b156e-c723-400a-b76e-4d914ad58712","Type":"ContainerStarted","Data":"e34411db6f73291ed53068ec6770a96ba2effb9046e0e8bb714c57b2c6703391"} Nov 04 05:43:12.196228 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:12.195993 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:12.199049 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:12.199018 4859 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Nov 04 05:43:12.199313 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:12.199278 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" podUID="7e5b156e-c723-400a-b76e-4d914ad58712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Nov 04 05:43:12.319924 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:12.319844 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" podStartSLOduration=2.319822816 podStartE2EDuration="2.319822816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:43:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:43:12.272395206 +0000 UTC m=+962.274030932" watchObservedRunningTime="2026-11-04 05:43:12.319822816 +0000 UTC m=+962.321458542" Nov 04 05:43:13.201689 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:13.199972 4859 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Nov 04 05:43:13.201689 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:13.200027 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" podUID="7e5b156e-c723-400a-b76e-4d914ad58712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Nov 04 05:43:14.197687 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:14.196971 4859 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Nov 04 05:43:14.197687 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:14.197028 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" podUID="7e5b156e-c723-400a-b76e-4d914ad58712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Nov 04 05:43:15.196688 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:15.196078 4859 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Nov 04 05:43:15.196688 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:15.196171 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" podUID="7e5b156e-c723-400a-b76e-4d914ad58712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Nov 04 05:43:16.200069 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:16.200030 4859 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Nov 04 05:43:16.200650 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:16.200617 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" podUID="7e5b156e-c723-400a-b76e-4d914ad58712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Nov 04 05:43:17.199688 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:17.199627 4859 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Nov 04 05:43:17.199901 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:17.199704 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" podUID="7e5b156e-c723-400a-b76e-4d914ad58712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Nov 04 05:43:18.354857 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:18.354824 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:18.355812 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:18.355782 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:18.356758 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:18.356738 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:43:28.341765 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:43:28.341728 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:43:32.401366 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.401334 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4"] Nov 04 05:43:32.404978 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.404962 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.463003 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.462968 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-a84d48\"/\"kube-root-ca.crt\"" Nov 04 05:43:32.473995 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.473968 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-model-cache\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.474156 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.474002 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-tmp-dir\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.474156 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.474078 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-dshm\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.474156 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.474111 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-home\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.474156 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.474140 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnh4r\" (UniqueName: \"kubernetes.io/projected/6283c937-1f92-49c6-a17b-7cf069384263-kube-api-access-rnh4r\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.474431 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.474160 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6283c937-1f92-49c6-a17b-7cf069384263-tls-certs\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.517602 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.517570 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-a84d48\"/\"openshift-service-ca.crt\"" Nov 04 05:43:32.517783 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.517611 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-a84d48\"/\"default-dockercfg-dvnsx\"" Nov 04 05:43:32.517783 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.517654 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-a84d48\"/\"distinct-w3-a84d48-kserve-self-signed-certs\"" Nov 04 05:43:32.520089 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.520063 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4"] Nov 04 05:43:32.576413 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.576377 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-model-cache\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.576616 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.576423 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-tmp-dir\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.576616 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.576466 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-dshm\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.576616 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.576506 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-home\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.576616 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.576559 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnh4r\" (UniqueName: \"kubernetes.io/projected/6283c937-1f92-49c6-a17b-7cf069384263-kube-api-access-rnh4r\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.576616 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.576588 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6283c937-1f92-49c6-a17b-7cf069384263-tls-certs\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.576928 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.576900 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-model-cache\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.577588 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.577563 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-home\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.577717 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.577646 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-tmp-dir\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.579971 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.579950 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-dshm\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.580255 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.580237 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6283c937-1f92-49c6-a17b-7cf069384263-tls-certs\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.643464 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.643435 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnh4r\" (UniqueName: \"kubernetes.io/projected/6283c937-1f92-49c6-a17b-7cf069384263-kube-api-access-rnh4r\") pod \"distinct-w3-a84d48-kserve-79dbd88975-zr6l4\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:32.717777 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:32.717756 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:33.112830 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:33.112757 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4"] Nov 04 05:43:33.116014 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:43:33.115992 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6283c937_1f92_49c6_a17b_7cf069384263.slice/crio-8a6d085db00b459375d9eaa6c35f480c6ac2f9829e95cb78eade579a2a3edd45 WatchSource:0}: Error finding container 8a6d085db00b459375d9eaa6c35f480c6ac2f9829e95cb78eade579a2a3edd45: Status 404 returned error can't find the container with id 8a6d085db00b459375d9eaa6c35f480c6ac2f9829e95cb78eade579a2a3edd45 Nov 04 05:43:33.269991 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:33.269954 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" event={"ID":"6283c937-1f92-49c6-a17b-7cf069384263","Type":"ContainerStarted","Data":"7147a2b5cd95210ee0dce5b6363b903a3a2d01741e0d71c475a22f308f1d7dc4"} Nov 04 05:43:33.270169 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:33.269994 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" event={"ID":"6283c937-1f92-49c6-a17b-7cf069384263","Type":"ContainerStarted","Data":"8a6d085db00b459375d9eaa6c35f480c6ac2f9829e95cb78eade579a2a3edd45"} Nov 04 05:43:33.396816 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:33.396726 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" podStartSLOduration=1.39671163 podStartE2EDuration="1.39671163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:43:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:43:33.39529265 +0000 UTC m=+983.396928375" watchObservedRunningTime="2026-11-04 05:43:33.39671163 +0000 UTC m=+983.398347354" Nov 04 05:43:33.718931 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:33.718903 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:36.335850 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.335822 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-rjx94-bzck8"] Nov 04 05:43:36.339895 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.339878 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rjx94-bzck8" Nov 04 05:43:36.403178 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.403156 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-rjx94-bzck8"] Nov 04 05:43:36.413877 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.413852 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5n2q\" (UniqueName: \"kubernetes.io/projected/768e34fb-e3f3-4b22-9012-1c8f11b38b76-kube-api-access-f5n2q\") pod \"periodic-gathering-rjx94-bzck8\" (UID: \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\") " pod="openshift-insights/periodic-gathering-rjx94-bzck8" Nov 04 05:43:36.414015 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.413996 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768e34fb-e3f3-4b22-9012-1c8f11b38b76-trusted-ca-bundle\") pod \"periodic-gathering-rjx94-bzck8\" (UID: \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\") " pod="openshift-insights/periodic-gathering-rjx94-bzck8" Nov 04 05:43:36.414141 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.414128 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768e34fb-e3f3-4b22-9012-1c8f11b38b76-service-ca-bundle\") pod \"periodic-gathering-rjx94-bzck8\" (UID: \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\") " pod="openshift-insights/periodic-gathering-rjx94-bzck8" Nov 04 05:43:36.414207 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.414192 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/768e34fb-e3f3-4b22-9012-1c8f11b38b76-archives-path\") pod \"periodic-gathering-rjx94-bzck8\" (UID: \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\") " pod="openshift-insights/periodic-gathering-rjx94-bzck8" Nov 04 05:43:36.454456 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.454434 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-z7zhw\"" Nov 04 05:43:36.515556 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.515526 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768e34fb-e3f3-4b22-9012-1c8f11b38b76-service-ca-bundle\") pod \"periodic-gathering-rjx94-bzck8\" (UID: \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\") " pod="openshift-insights/periodic-gathering-rjx94-bzck8" Nov 04 05:43:36.515700 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.515581 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/768e34fb-e3f3-4b22-9012-1c8f11b38b76-archives-path\") pod \"periodic-gathering-rjx94-bzck8\" (UID: \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\") " pod="openshift-insights/periodic-gathering-rjx94-bzck8" Nov 04 05:43:36.515700 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.515617 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5n2q\" (UniqueName: \"kubernetes.io/projected/768e34fb-e3f3-4b22-9012-1c8f11b38b76-kube-api-access-f5n2q\") pod \"periodic-gathering-rjx94-bzck8\" (UID: \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\") " pod="openshift-insights/periodic-gathering-rjx94-bzck8" Nov 04 05:43:36.515781 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.515698 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768e34fb-e3f3-4b22-9012-1c8f11b38b76-trusted-ca-bundle\") pod \"periodic-gathering-rjx94-bzck8\" (UID: \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\") " pod="openshift-insights/periodic-gathering-rjx94-bzck8" Nov 04 05:43:36.516038 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.516012 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/768e34fb-e3f3-4b22-9012-1c8f11b38b76-archives-path\") pod \"periodic-gathering-rjx94-bzck8\" (UID: \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\") " pod="openshift-insights/periodic-gathering-rjx94-bzck8" Nov 04 05:43:36.516289 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.516271 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768e34fb-e3f3-4b22-9012-1c8f11b38b76-service-ca-bundle\") pod \"periodic-gathering-rjx94-bzck8\" (UID: \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\") " pod="openshift-insights/periodic-gathering-rjx94-bzck8" Nov 04 05:43:36.516390 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.516373 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768e34fb-e3f3-4b22-9012-1c8f11b38b76-trusted-ca-bundle\") pod \"periodic-gathering-rjx94-bzck8\" (UID: \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\") " pod="openshift-insights/periodic-gathering-rjx94-bzck8" Nov 04 05:43:36.582784 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.582753 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5n2q\" (UniqueName: \"kubernetes.io/projected/768e34fb-e3f3-4b22-9012-1c8f11b38b76-kube-api-access-f5n2q\") pod \"periodic-gathering-rjx94-bzck8\" (UID: \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\") " pod="openshift-insights/periodic-gathering-rjx94-bzck8" Nov 04 05:43:36.652223 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:36.652154 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rjx94-bzck8" Nov 04 05:43:37.040080 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:37.040049 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-rjx94-bzck8"] Nov 04 05:43:37.043837 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:43:37.043809 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod768e34fb_e3f3_4b22_9012_1c8f11b38b76.slice/crio-13ecc4d82d3985ebbba26cec9d2b05d6477055eaf42a6a12db904d7c18559dfa WatchSource:0}: Error finding container 13ecc4d82d3985ebbba26cec9d2b05d6477055eaf42a6a12db904d7c18559dfa: Status 404 returned error can't find the container with id 13ecc4d82d3985ebbba26cec9d2b05d6477055eaf42a6a12db904d7c18559dfa Nov 04 05:43:37.293984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:37.293902 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rjx94-bzck8" event={"ID":"768e34fb-e3f3-4b22-9012-1c8f11b38b76","Type":"ContainerStarted","Data":"1612d4e70cb6d164e1a47fc5c6b90f6cc12f00b5045024a51a8b378bb7bc1b29"} Nov 04 05:43:37.293984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:37.293951 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rjx94-bzck8" event={"ID":"768e34fb-e3f3-4b22-9012-1c8f11b38b76","Type":"ContainerStarted","Data":"13ecc4d82d3985ebbba26cec9d2b05d6477055eaf42a6a12db904d7c18559dfa"} Nov 04 05:43:37.419924 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:37.419875 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-rjx94-bzck8" podStartSLOduration=1.419860668 podStartE2EDuration="1.419860668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:43:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:43:37.418541665 +0000 UTC m=+987.420177390" watchObservedRunningTime="2026-11-04 05:43:37.419860668 +0000 UTC m=+987.421496391" Nov 04 05:43:42.718866 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:42.718830 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:42.731425 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:42.731399 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:43.339377 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:43:43.339348 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:43:58.342370 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:43:58.342331 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:44:05.688265 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:05.688228 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7200-090f0b"] Nov 04 05:44:05.694155 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:05.694130 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7200-090f0b" Nov 04 05:44:05.729038 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:05.729011 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxwpq\" (UniqueName: \"kubernetes.io/projected/64e4dab1-1995-45bf-95bc-5c8dc7bf265b-kube-api-access-rxwpq\") pod \"test-curl-7200-090f0b\" (UID: \"64e4dab1-1995-45bf-95bc-5c8dc7bf265b\") " pod="opendatahub/test-curl-7200-090f0b" Nov 04 05:44:05.752270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:05.752240 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7200-090f0b"] Nov 04 05:44:05.829887 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:05.829847 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxwpq\" (UniqueName: \"kubernetes.io/projected/64e4dab1-1995-45bf-95bc-5c8dc7bf265b-kube-api-access-rxwpq\") pod \"test-curl-7200-090f0b\" (UID: \"64e4dab1-1995-45bf-95bc-5c8dc7bf265b\") " pod="opendatahub/test-curl-7200-090f0b" Nov 04 05:44:05.868998 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:05.868959 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lxtlf\"" Nov 04 05:44:05.895486 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:05.895451 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxwpq\" (UniqueName: \"kubernetes.io/projected/64e4dab1-1995-45bf-95bc-5c8dc7bf265b-kube-api-access-rxwpq\") pod \"test-curl-7200-090f0b\" (UID: \"64e4dab1-1995-45bf-95bc-5c8dc7bf265b\") " pod="opendatahub/test-curl-7200-090f0b" Nov 04 05:44:06.011136 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:06.011054 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7200-090f0b" Nov 04 05:44:06.484949 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:06.484919 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7200-090f0b"] Nov 04 05:44:06.492602 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:06.492577 4859 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:44:07.494184 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:07.494151 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7200-090f0b" event={"ID":"64e4dab1-1995-45bf-95bc-5c8dc7bf265b","Type":"ContainerStarted","Data":"a948164b4d92ca59cf9b42fa5b854b39b64015b37a3043625e2f4811b80b2802"} Nov 04 05:44:08.501095 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:08.501064 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7200-090f0b" event={"ID":"64e4dab1-1995-45bf-95bc-5c8dc7bf265b","Type":"ContainerStarted","Data":"d073f575e0c6026626a3b9425da4b9ebf7718b2a71c515d0c557a7ca1a588c19"} Nov 04 05:44:08.626393 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:08.626334 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7200-090f0b" podStartSLOduration=2.172551327 podStartE2EDuration="3.626315789s" totalImagesPullingTime="1.453764462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:08.623356664 +0000 UTC m=+1018.624992389" watchObservedRunningTime="2026-11-04 05:44:08.626315789 +0000 UTC m=+1018.627951518" Nov 04 05:44:10.183893 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:10.183860 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7200-090f0b"] Nov 04 05:44:10.186747 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:10.186724 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7200-090f0b"] Nov 04 05:44:10.512628 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:10.512510 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7200-090f0b" podUID="64e4dab1-1995-45bf-95bc-5c8dc7bf265b" containerName="test-curl-7200-090f0b" containerID="cri-o://d073f575e0c6026626a3b9425da4b9ebf7718b2a71c515d0c557a7ca1a588c19" gracePeriod=2 Nov 04 05:44:10.568242 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:44:10.568202 4859 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7200-090f0b\" is forbidden: User \"system:node:ip-10-0-133-143.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-143.us-west-2.compute.internal' and this object" podUID="64e4dab1-1995-45bf-95bc-5c8dc7bf265b" pod="opendatahub/test-curl-7200-090f0b" Nov 04 05:44:12.766171 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:12.766119 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7200-090f0b" Nov 04 05:44:12.831797 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:12.831775 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64e4dab1-1995-45bf-95bc-5c8dc7bf265b-kube-api-access-rxwpq\" (UniqueName: \"kubernetes.io/projected/64e4dab1-1995-45bf-95bc-5c8dc7bf265b-kube-api-access-rxwpq\") pod \"64e4dab1-1995-45bf-95bc-5c8dc7bf265b\" (UID: \"64e4dab1-1995-45bf-95bc-5c8dc7bf265b\") " Nov 04 05:44:12.833858 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:12.833835 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64e4dab1-1995-45bf-95bc-5c8dc7bf265b-kube-api-access-rxwpq" pod "64e4dab1-1995-45bf-95bc-5c8dc7bf265b" (UID: "64e4dab1-1995-45bf-95bc-5c8dc7bf265b"). InnerVolumeSpecName "kube-api-access-rxwpq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:12.932644 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:12.932619 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxwpq\" (UniqueName: \"kubernetes.io/projected/64e4dab1-1995-45bf-95bc-5c8dc7bf265b-kube-api-access-rxwpq\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:13.532515 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:13.532478 4859 generic.go:356] "Generic (PLEG): container finished" podID="64e4dab1-1995-45bf-95bc-5c8dc7bf265b" containerID="d073f575e0c6026626a3b9425da4b9ebf7718b2a71c515d0c557a7ca1a588c19" exitCode=137 Nov 04 05:44:13.532515 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:13.532512 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7200-090f0b" Nov 04 05:44:13.532752 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:13.532618 4859 scope.go:122] "RemoveContainer" containerID="d073f575e0c6026626a3b9425da4b9ebf7718b2a71c515d0c557a7ca1a588c19" Nov 04 05:44:13.541789 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:13.541768 4859 scope.go:122] "RemoveContainer" containerID="d073f575e0c6026626a3b9425da4b9ebf7718b2a71c515d0c557a7ca1a588c19" Nov 04 05:44:13.542056 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:44:13.542034 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d073f575e0c6026626a3b9425da4b9ebf7718b2a71c515d0c557a7ca1a588c19\": container with ID starting with d073f575e0c6026626a3b9425da4b9ebf7718b2a71c515d0c557a7ca1a588c19 not found: ID does not exist" containerID="d073f575e0c6026626a3b9425da4b9ebf7718b2a71c515d0c557a7ca1a588c19" Nov 04 05:44:13.542120 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:13.542065 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d073f575e0c6026626a3b9425da4b9ebf7718b2a71c515d0c557a7ca1a588c19"} err="failed to get container status \"d073f575e0c6026626a3b9425da4b9ebf7718b2a71c515d0c557a7ca1a588c19\": rpc error: code = NotFound desc = could not find container \"d073f575e0c6026626a3b9425da4b9ebf7718b2a71c515d0c557a7ca1a588c19\": container with ID starting with d073f575e0c6026626a3b9425da4b9ebf7718b2a71c515d0c557a7ca1a588c19 not found: ID does not exist" Nov 04 05:44:13.596584 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:44:13.596554 4859 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7200-090f0b\" is forbidden: User \"system:node:ip-10-0-133-143.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-143.us-west-2.compute.internal' and this object" podUID="64e4dab1-1995-45bf-95bc-5c8dc7bf265b" pod="opendatahub/test-curl-7200-090f0b" Nov 04 05:44:14.115855 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:14.115821 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="64e4dab1-1995-45bf-95bc-5c8dc7bf265b" path="/var/lib/kubelet/pods/64e4dab1-1995-45bf-95bc-5c8dc7bf265b/volumes" Nov 04 05:44:17.537049 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.537016 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8"] Nov 04 05:44:17.537542 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.537528 4859 cpu_manager.go:414] "containerMap: removing container" podUID="64e4dab1-1995-45bf-95bc-5c8dc7bf265b" containerName="test-curl-7200-090f0b" Nov 04 05:44:17.537585 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.537543 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64e4dab1-1995-45bf-95bc-5c8dc7bf265b" containerName="test-curl-7200-090f0b" Nov 04 05:44:17.537693 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.537682 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="64e4dab1-1995-45bf-95bc-5c8dc7bf265b" containerName="test-curl-7200-090f0b" Nov 04 05:44:17.542215 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.542198 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.580319 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.580284 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-home\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.580454 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.580335 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-model-cache\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.580512 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.580475 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-dshm\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.580550 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.580516 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26b961ed-a12f-4a67-80c9-d41e525e38f8-tls-certs\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.580636 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.580549 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmmmp\" (UniqueName: \"kubernetes.io/projected/26b961ed-a12f-4a67-80c9-d41e525e38f8-kube-api-access-nmmmp\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.580636 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.580627 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-tmp-dir\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.599340 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.599314 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-961485\"/\"kube-root-ca.crt\"" Nov 04 05:44:17.628932 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.628899 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-961485\"/\"openshift-service-ca.crt\"" Nov 04 05:44:17.653505 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.653477 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-961485\"/\"distinct-w2-961485-kserve-self-signed-certs\"" Nov 04 05:44:17.653505 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.653488 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-961485\"/\"default-dockercfg-mzgh6\"" Nov 04 05:44:17.657489 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.657467 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8"] Nov 04 05:44:17.681812 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.681785 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-tmp-dir\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.681935 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.681923 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-home\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.681997 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.681953 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-model-cache\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.682043 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.682009 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-dshm\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.682043 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.682036 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26b961ed-a12f-4a67-80c9-d41e525e38f8-tls-certs\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.682134 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.682075 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmmmp\" (UniqueName: \"kubernetes.io/projected/26b961ed-a12f-4a67-80c9-d41e525e38f8-kube-api-access-nmmmp\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.682230 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.682209 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-tmp-dir\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.682466 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.682446 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-model-cache\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.682701 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.682660 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-home\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.688648 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.688620 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-dshm\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.704739 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.704649 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26b961ed-a12f-4a67-80c9-d41e525e38f8-tls-certs\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.747259 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.747223 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmmmp\" (UniqueName: \"kubernetes.io/projected/26b961ed-a12f-4a67-80c9-d41e525e38f8-kube-api-access-nmmmp\") pod \"distinct-w2-961485-kserve-dbf9df886-2z9h8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:17.863847 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:17.863767 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:18.018330 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:18.018296 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc"] Nov 04 05:44:18.280683 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:18.280638 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8"] Nov 04 05:44:18.284114 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:44:18.284088 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26b961ed_a12f_4a67_80c9_d41e525e38f8.slice/crio-4cde191106a1c0b4498e83d2e76c9a1b4c26aa9b2689862cae5fa3d81c578b50 WatchSource:0}: Error finding container 4cde191106a1c0b4498e83d2e76c9a1b4c26aa9b2689862cae5fa3d81c578b50: Status 404 returned error can't find the container with id 4cde191106a1c0b4498e83d2e76c9a1b4c26aa9b2689862cae5fa3d81c578b50 Nov 04 05:44:18.563702 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:18.563606 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" event={"ID":"26b961ed-a12f-4a67-80c9-d41e525e38f8","Type":"ContainerStarted","Data":"8a9871d9dc44094ef997c78e252e57f99b57170fd5013021c5b05f186cfecca5"} Nov 04 05:44:18.563702 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:18.563654 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" event={"ID":"26b961ed-a12f-4a67-80c9-d41e525e38f8","Type":"ContainerStarted","Data":"4cde191106a1c0b4498e83d2e76c9a1b4c26aa9b2689862cae5fa3d81c578b50"} Nov 04 05:44:18.690924 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:18.690872 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" podStartSLOduration=1.6908580130000002 podStartE2EDuration="1.690858013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:18.688164615 +0000 UTC m=+1028.689800341" watchObservedRunningTime="2026-11-04 05:44:18.690858013 +0000 UTC m=+1028.692493738" Nov 04 05:44:18.864958 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:18.864877 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:23.440294 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:23.440208 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6"] Nov 04 05:44:27.864264 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:27.864224 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:27.877088 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:27.877064 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:28.341805 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:44:28.341770 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:44:28.640589 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:28.640514 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:44:33.039061 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.038999 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" podUID="b34a4fed-23f4-456d-86a7-828eaf8c8bbc" containerName="main" containerID="cri-o://6de8ebc46daed2d9c1b7dd0a3b2eea2c15d3203e3aaf7e141485dd118309429b" gracePeriod=45 Nov 04 05:44:33.243120 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.243077 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" podUID="b34a4fed-23f4-456d-86a7-828eaf8c8bbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": dial tcp 10.132.0.59:8000: connect: connection refused" Nov 04 05:44:33.342968 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.342944 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:44:33.459840 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.459806 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-tmp-dir\") pod \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " Nov 04 05:44:33.460024 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.459875 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-tls-certs\" (UniqueName: \"kubernetes.io/secret/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-tls-certs\") pod \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " Nov 04 05:44:33.460024 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.459944 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-dshm\") pod \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " Nov 04 05:44:33.460143 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.460036 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-home\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-home\") pod \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " Nov 04 05:44:33.460143 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.460079 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-model-cache\") pod \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " Nov 04 05:44:33.460143 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.460095 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-tmp-dir" pod "b34a4fed-23f4-456d-86a7-828eaf8c8bbc" (UID: "b34a4fed-23f4-456d-86a7-828eaf8c8bbc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:33.460296 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.460175 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-kube-api-access-z4s2l\" (UniqueName: \"kubernetes.io/projected/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-kube-api-access-z4s2l\") pod \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\" (UID: \"b34a4fed-23f4-456d-86a7-828eaf8c8bbc\") " Nov 04 05:44:33.460348 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.460326 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-home" pod "b34a4fed-23f4-456d-86a7-828eaf8c8bbc" (UID: "b34a4fed-23f4-456d-86a7-828eaf8c8bbc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:33.460401 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.460380 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-model-cache" pod "b34a4fed-23f4-456d-86a7-828eaf8c8bbc" (UID: "b34a4fed-23f4-456d-86a7-828eaf8c8bbc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:33.460700 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.460656 4859 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-tmp-dir\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:33.460700 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.460696 4859 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-home\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:33.460889 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.460710 4859 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-model-cache\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:33.462805 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.462779 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-dshm" pod "b34a4fed-23f4-456d-86a7-828eaf8c8bbc" (UID: "b34a4fed-23f4-456d-86a7-828eaf8c8bbc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:33.462805 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.462786 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-kube-api-access-z4s2l" pod "b34a4fed-23f4-456d-86a7-828eaf8c8bbc" (UID: "b34a4fed-23f4-456d-86a7-828eaf8c8bbc"). InnerVolumeSpecName "kube-api-access-z4s2l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:33.463056 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.463041 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-tls-certs" pod "b34a4fed-23f4-456d-86a7-828eaf8c8bbc" (UID: "b34a4fed-23f4-456d-86a7-828eaf8c8bbc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:33.561898 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.561869 4859 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-tls-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:33.561898 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.561894 4859 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-dshm\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:33.562073 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.561904 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4s2l\" (UniqueName: \"kubernetes.io/projected/b34a4fed-23f4-456d-86a7-828eaf8c8bbc-kube-api-access-z4s2l\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:33.654646 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.654554 4859 generic.go:356] "Generic (PLEG): container finished" podID="b34a4fed-23f4-456d-86a7-828eaf8c8bbc" containerID="6de8ebc46daed2d9c1b7dd0a3b2eea2c15d3203e3aaf7e141485dd118309429b" exitCode=0 Nov 04 05:44:33.654646 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.654593 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" Nov 04 05:44:33.654900 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.654640 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" event={"ID":"b34a4fed-23f4-456d-86a7-828eaf8c8bbc","Type":"ContainerDied","Data":"6de8ebc46daed2d9c1b7dd0a3b2eea2c15d3203e3aaf7e141485dd118309429b"} Nov 04 05:44:33.654900 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.654701 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc" event={"ID":"b34a4fed-23f4-456d-86a7-828eaf8c8bbc","Type":"ContainerDied","Data":"c58cfca1058c60c6ede38818457b968bd50de66addbe5c4e9c050c55eba6bf38"} Nov 04 05:44:33.654900 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.654726 4859 scope.go:122] "RemoveContainer" containerID="6de8ebc46daed2d9c1b7dd0a3b2eea2c15d3203e3aaf7e141485dd118309429b" Nov 04 05:44:33.665056 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.665040 4859 scope.go:122] "RemoveContainer" containerID="6de8ebc46daed2d9c1b7dd0a3b2eea2c15d3203e3aaf7e141485dd118309429b" Nov 04 05:44:33.665325 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:44:33.665305 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6de8ebc46daed2d9c1b7dd0a3b2eea2c15d3203e3aaf7e141485dd118309429b\": container with ID starting with 6de8ebc46daed2d9c1b7dd0a3b2eea2c15d3203e3aaf7e141485dd118309429b not found: ID does not exist" containerID="6de8ebc46daed2d9c1b7dd0a3b2eea2c15d3203e3aaf7e141485dd118309429b" Nov 04 05:44:33.665423 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.665331 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6de8ebc46daed2d9c1b7dd0a3b2eea2c15d3203e3aaf7e141485dd118309429b"} err="failed to get container status \"6de8ebc46daed2d9c1b7dd0a3b2eea2c15d3203e3aaf7e141485dd118309429b\": rpc error: code = NotFound desc = could not find container \"6de8ebc46daed2d9c1b7dd0a3b2eea2c15d3203e3aaf7e141485dd118309429b\": container with ID starting with 6de8ebc46daed2d9c1b7dd0a3b2eea2c15d3203e3aaf7e141485dd118309429b not found: ID does not exist" Nov 04 05:44:33.832195 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.832161 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc"] Nov 04 05:44:33.839609 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:33.839583 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-283815/simulator-w5-283815-kserve-549cccb89-rrrnc"] Nov 04 05:44:34.115502 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:34.115468 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b34a4fed-23f4-456d-86a7-828eaf8c8bbc" path="/var/lib/kubelet/pods/b34a4fed-23f4-456d-86a7-828eaf8c8bbc/volumes" Nov 04 05:44:36.624594 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:36.624562 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f"] Nov 04 05:44:38.460617 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.460575 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" podUID="824e4ea9-5112-4827-9392-9b22b87e3a58" containerName="main" containerID="cri-o://7f607cbbaa0de218cb5049118926da6f5b301cc140dba75869c09e3e9074aa08" gracePeriod=45 Nov 04 05:44:38.696457 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.696418 4859 generic.go:356] "Generic (PLEG): container finished" podID="824e4ea9-5112-4827-9392-9b22b87e3a58" containerID="7f607cbbaa0de218cb5049118926da6f5b301cc140dba75869c09e3e9074aa08" exitCode=0 Nov 04 05:44:38.696637 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.696495 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" event={"ID":"824e4ea9-5112-4827-9392-9b22b87e3a58","Type":"ContainerDied","Data":"7f607cbbaa0de218cb5049118926da6f5b301cc140dba75869c09e3e9074aa08"} Nov 04 05:44:38.755227 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.755204 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:44:38.915621 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.915591 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-home\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-home\") pod \"824e4ea9-5112-4827-9392-9b22b87e3a58\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " Nov 04 05:44:38.915824 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.915649 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-model-cache\") pod \"824e4ea9-5112-4827-9392-9b22b87e3a58\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " Nov 04 05:44:38.915824 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.915703 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-dshm\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-dshm\") pod \"824e4ea9-5112-4827-9392-9b22b87e3a58\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " Nov 04 05:44:38.915824 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.915799 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/824e4ea9-5112-4827-9392-9b22b87e3a58-kube-api-access-8wn2h\" (UniqueName: \"kubernetes.io/projected/824e4ea9-5112-4827-9392-9b22b87e3a58-kube-api-access-8wn2h\") pod \"824e4ea9-5112-4827-9392-9b22b87e3a58\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " Nov 04 05:44:38.916002 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.915865 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-tmp-dir\") pod \"824e4ea9-5112-4827-9392-9b22b87e3a58\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " Nov 04 05:44:38.916002 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.915929 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-home" pod "824e4ea9-5112-4827-9392-9b22b87e3a58" (UID: "824e4ea9-5112-4827-9392-9b22b87e3a58"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:38.916002 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.915943 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-model-cache" pod "824e4ea9-5112-4827-9392-9b22b87e3a58" (UID: "824e4ea9-5112-4827-9392-9b22b87e3a58"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:38.916002 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.915960 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/824e4ea9-5112-4827-9392-9b22b87e3a58-tls-certs\" (UniqueName: \"kubernetes.io/secret/824e4ea9-5112-4827-9392-9b22b87e3a58-tls-certs\") pod \"824e4ea9-5112-4827-9392-9b22b87e3a58\" (UID: \"824e4ea9-5112-4827-9392-9b22b87e3a58\") " Nov 04 05:44:38.916212 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.916156 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-tmp-dir" pod "824e4ea9-5112-4827-9392-9b22b87e3a58" (UID: "824e4ea9-5112-4827-9392-9b22b87e3a58"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:38.916466 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.916444 4859 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-tmp-dir\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:38.916552 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.916473 4859 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-home\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:38.916552 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.916486 4859 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-model-cache\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:38.918486 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.918452 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/824e4ea9-5112-4827-9392-9b22b87e3a58-tls-certs" pod "824e4ea9-5112-4827-9392-9b22b87e3a58" (UID: "824e4ea9-5112-4827-9392-9b22b87e3a58"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:38.918611 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.918497 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-dshm" pod "824e4ea9-5112-4827-9392-9b22b87e3a58" (UID: "824e4ea9-5112-4827-9392-9b22b87e3a58"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:38.918611 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:38.918503 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/824e4ea9-5112-4827-9392-9b22b87e3a58-kube-api-access-8wn2h" pod "824e4ea9-5112-4827-9392-9b22b87e3a58" (UID: "824e4ea9-5112-4827-9392-9b22b87e3a58"). InnerVolumeSpecName "kube-api-access-8wn2h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:39.017340 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:39.017264 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wn2h\" (UniqueName: \"kubernetes.io/projected/824e4ea9-5112-4827-9392-9b22b87e3a58-kube-api-access-8wn2h\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:39.017340 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:39.017296 4859 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/824e4ea9-5112-4827-9392-9b22b87e3a58-tls-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:39.017340 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:39.017306 4859 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/824e4ea9-5112-4827-9392-9b22b87e3a58-dshm\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:39.703336 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:39.703304 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" Nov 04 05:44:39.703789 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:39.703351 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6" event={"ID":"824e4ea9-5112-4827-9392-9b22b87e3a58","Type":"ContainerDied","Data":"509716d5e177229b85eb55645787115c708a6d3a59f537ede30be96b9bd8aec9"} Nov 04 05:44:39.703789 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:39.703401 4859 scope.go:122] "RemoveContainer" containerID="7f607cbbaa0de218cb5049118926da6f5b301cc140dba75869c09e3e9074aa08" Nov 04 05:44:39.880223 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:39.880195 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6"] Nov 04 05:44:39.934073 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:39.934038 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-fbbad9/premium-w6-fbbad9-kserve-7cf97f9ccd-gzgz6"] Nov 04 05:44:40.115919 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:40.115852 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="824e4ea9-5112-4827-9392-9b22b87e3a58" path="/var/lib/kubelet/pods/824e4ea9-5112-4827-9392-9b22b87e3a58/volumes" Nov 04 05:44:41.625363 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:41.625293 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" podUID="27f04fa4-41f7-4988-b5a4-d81187643cf7" containerName="payload-pre-processing" containerID="cri-o://1cc40cae53d8feb15552a5ed5847eda217a1c90c06c65d752cd5a135b111a7d5" gracePeriod=25 Nov 04 05:44:42.575880 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:42.575836 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" podUID="27f04fa4-41f7-4988-b5a4-d81187643cf7" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:44:44.057181 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:44.057142 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj"] Nov 04 05:44:49.057677 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:49.057625 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" podUID="082c0bec-9706-4234-8ce4-e57f08aa23f5" containerName="payload-pre-processing" containerID="cri-o://13d852f8e60388b7dc548e556681d6fa5d09f80bf28c08f82594b6afad1b3b8e" gracePeriod=25 Nov 04 05:44:51.598838 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:51.598807 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr"] Nov 04 05:44:51.599301 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:51.598990 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" podUID="22dcf276-7849-4247-89d2-98a53b55865b" containerName="maas-api" containerID="cri-o://93088e851cc6b7398069f10327690909bc7150673f27b5058e63f86ede4b8233" gracePeriod=30 Nov 04 05:44:51.791744 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:51.791715 4859 generic.go:356] "Generic (PLEG): container finished" podID="22dcf276-7849-4247-89d2-98a53b55865b" containerID="93088e851cc6b7398069f10327690909bc7150673f27b5058e63f86ede4b8233" exitCode=0 Nov 04 05:44:51.791906 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:51.791784 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" event={"ID":"22dcf276-7849-4247-89d2-98a53b55865b","Type":"ContainerDied","Data":"93088e851cc6b7398069f10327690909bc7150673f27b5058e63f86ede4b8233"} Nov 04 05:44:51.883744 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:51.883712 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:44:51.961740 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:51.961706 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22dcf276-7849-4247-89d2-98a53b55865b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22dcf276-7849-4247-89d2-98a53b55865b-openshift-service-ca\") pod \"22dcf276-7849-4247-89d2-98a53b55865b\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " Nov 04 05:44:51.961901 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:51.961748 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-metrics-certs\") pod \"22dcf276-7849-4247-89d2-98a53b55865b\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " Nov 04 05:44:51.961901 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:51.961784 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-maas-api-tls\") pod \"22dcf276-7849-4247-89d2-98a53b55865b\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " Nov 04 05:44:51.961901 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:51.961828 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22dcf276-7849-4247-89d2-98a53b55865b-kube-api-access-bf975\" (UniqueName: \"kubernetes.io/projected/22dcf276-7849-4247-89d2-98a53b55865b-kube-api-access-bf975\") pod \"22dcf276-7849-4247-89d2-98a53b55865b\" (UID: \"22dcf276-7849-4247-89d2-98a53b55865b\") " Nov 04 05:44:51.962075 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:51.962049 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22dcf276-7849-4247-89d2-98a53b55865b-openshift-service-ca" pod "22dcf276-7849-4247-89d2-98a53b55865b" (UID: "22dcf276-7849-4247-89d2-98a53b55865b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:51.962246 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:51.962226 4859 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22dcf276-7849-4247-89d2-98a53b55865b-openshift-service-ca\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:51.963933 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:51.963905 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22dcf276-7849-4247-89d2-98a53b55865b-kube-api-access-bf975" pod "22dcf276-7849-4247-89d2-98a53b55865b" (UID: "22dcf276-7849-4247-89d2-98a53b55865b"). InnerVolumeSpecName "kube-api-access-bf975". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:51.964307 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:51.964286 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-metrics-certs" pod "22dcf276-7849-4247-89d2-98a53b55865b" (UID: "22dcf276-7849-4247-89d2-98a53b55865b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:51.964377 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:51.964308 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-maas-api-tls" pod "22dcf276-7849-4247-89d2-98a53b55865b" (UID: "22dcf276-7849-4247-89d2-98a53b55865b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:52.063517 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:52.063481 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bf975\" (UniqueName: \"kubernetes.io/projected/22dcf276-7849-4247-89d2-98a53b55865b-kube-api-access-bf975\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:52.063517 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:52.063514 4859 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-metrics-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:52.063517 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:52.063523 4859 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22dcf276-7849-4247-89d2-98a53b55865b-maas-api-tls\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:52.577000 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:52.576964 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" podUID="27f04fa4-41f7-4988-b5a4-d81187643cf7" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:44:52.807973 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:52.807942 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" Nov 04 05:44:52.807973 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:52.807959 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr" event={"ID":"22dcf276-7849-4247-89d2-98a53b55865b","Type":"ContainerDied","Data":"b52bb6a370b04de01e5259fcd5e223985b240948663c3a6c4e4ba1cc2073dbda"} Nov 04 05:44:52.808537 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:52.808027 4859 scope.go:122] "RemoveContainer" containerID="93088e851cc6b7398069f10327690909bc7150673f27b5058e63f86ede4b8233" Nov 04 05:44:53.016885 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:53.016847 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr"] Nov 04 05:44:53.070982 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:53.070944 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-90dc8d-784cbc88cf-s8rzr"] Nov 04 05:44:54.115881 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.115849 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22dcf276-7849-4247-89d2-98a53b55865b" path="/var/lib/kubelet/pods/22dcf276-7849-4247-89d2-98a53b55865b/volumes" Nov 04 05:44:54.732967 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.732933 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px"] Nov 04 05:44:54.734640 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.734613 4859 cpu_manager.go:414] "containerMap: removing container" podUID="824e4ea9-5112-4827-9392-9b22b87e3a58" containerName="main" Nov 04 05:44:54.734849 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.734834 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="824e4ea9-5112-4827-9392-9b22b87e3a58" containerName="main" Nov 04 05:44:54.734986 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.734976 4859 cpu_manager.go:414] "containerMap: removing container" podUID="22dcf276-7849-4247-89d2-98a53b55865b" containerName="maas-api" Nov 04 05:44:54.735072 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.735052 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22dcf276-7849-4247-89d2-98a53b55865b" containerName="maas-api" Nov 04 05:44:54.735179 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.735168 4859 cpu_manager.go:414] "containerMap: removing container" podUID="b34a4fed-23f4-456d-86a7-828eaf8c8bbc" containerName="main" Nov 04 05:44:54.735255 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.735246 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b34a4fed-23f4-456d-86a7-828eaf8c8bbc" containerName="main" Nov 04 05:44:54.735462 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.735452 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="22dcf276-7849-4247-89d2-98a53b55865b" containerName="maas-api" Nov 04 05:44:54.735547 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.735537 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="b34a4fed-23f4-456d-86a7-828eaf8c8bbc" containerName="main" Nov 04 05:44:54.735628 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.735619 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="824e4ea9-5112-4827-9392-9b22b87e3a58" containerName="main" Nov 04 05:44:54.747706 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.744919 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.802311 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.802283 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-fbbad9-openshift-default-dockercfg-nb2h5\"" Nov 04 05:44:54.835387 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.835342 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-istio-envoy\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.840707 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.840649 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svw7h\" (UniqueName: \"kubernetes.io/projected/650965ab-b041-4c27-ab58-45c70ec37f94-kube-api-access-svw7h\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.841162 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.841042 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/650965ab-b041-4c27-ab58-45c70ec37f94-istio-token\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.841458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.841437 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/650965ab-b041-4c27-ab58-45c70ec37f94-istio-podinfo\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.841733 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.841681 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-credential-socket\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.841872 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.841759 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-istio-data\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.841872 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.841826 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/650965ab-b041-4c27-ab58-45c70ec37f94-istiod-ca-cert\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.841872 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.841859 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-workload-certs\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.842067 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.842021 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-workload-socket\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.860421 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.860394 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px"] Nov 04 05:44:54.943739 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.943705 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/650965ab-b041-4c27-ab58-45c70ec37f94-istiod-ca-cert\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.943881 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.943753 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-workload-certs\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.943881 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.943786 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-workload-socket\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.943881 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.943830 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-istio-envoy\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.943881 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.943867 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svw7h\" (UniqueName: \"kubernetes.io/projected/650965ab-b041-4c27-ab58-45c70ec37f94-kube-api-access-svw7h\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.944082 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.943888 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/650965ab-b041-4c27-ab58-45c70ec37f94-istio-token\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.944082 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.943916 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/650965ab-b041-4c27-ab58-45c70ec37f94-istio-podinfo\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.944082 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.943999 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-credential-socket\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.944082 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.944059 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-istio-data\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.944295 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.944251 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-workload-socket\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.944356 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.944340 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-workload-certs\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.944472 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.944450 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-istio-data\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.944523 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.944478 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/650965ab-b041-4c27-ab58-45c70ec37f94-istiod-ca-cert\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.944555 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.944528 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-credential-socket\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.946182 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.946163 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-istio-envoy\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:54.946790 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:54.946773 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/650965ab-b041-4c27-ab58-45c70ec37f94-istio-podinfo\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:55.017401 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:55.017330 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/650965ab-b041-4c27-ab58-45c70ec37f94-istio-token\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:55.019138 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:55.019114 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svw7h\" (UniqueName: \"kubernetes.io/projected/650965ab-b041-4c27-ab58-45c70ec37f94-kube-api-access-svw7h\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:55.062278 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:55.062246 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:55.515420 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:55.515397 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px"] Nov 04 05:44:55.518248 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:44:55.518220 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod650965ab_b041_4c27_ab58_45c70ec37f94.slice/crio-65b9e5d8da52faf44e8fbecbe2dd500ac6a888f39a0bccadefd21e50bd1b98f4 WatchSource:0}: Error finding container 65b9e5d8da52faf44e8fbecbe2dd500ac6a888f39a0bccadefd21e50bd1b98f4: Status 404 returned error can't find the container with id 65b9e5d8da52faf44e8fbecbe2dd500ac6a888f39a0bccadefd21e50bd1b98f4 Nov 04 05:44:55.520599 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:55.520567 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:44:55.520756 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:55.520644 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:44:55.520756 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:55.520720 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:44:55.853275 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:55.853182 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" event={"ID":"650965ab-b041-4c27-ab58-45c70ec37f94","Type":"ContainerStarted","Data":"bd8ca6e751832d138d18b14304807bc375881cecc6cc9e6e33a4d79f5b09385c"} Nov 04 05:44:55.853275 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:55.853244 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" event={"ID":"650965ab-b041-4c27-ab58-45c70ec37f94","Type":"ContainerStarted","Data":"65b9e5d8da52faf44e8fbecbe2dd500ac6a888f39a0bccadefd21e50bd1b98f4"} Nov 04 05:44:56.063483 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:56.063446 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:44:56.065034 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:56.065007 4859 patch_prober.go:29] interesting pod/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:44:56.065167 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:56.065055 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" podUID="650965ab-b041-4c27-ab58-45c70ec37f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Nov 04 05:44:57.063081 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:57.063045 4859 patch_prober.go:29] interesting pod/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:44:57.063545 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:57.063108 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" podUID="650965ab-b041-4c27-ab58-45c70ec37f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Nov 04 05:44:58.063493 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:58.063462 4859 patch_prober.go:29] interesting pod/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:44:58.063884 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:58.063515 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" podUID="650965ab-b041-4c27-ab58-45c70ec37f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Nov 04 05:44:58.344692 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:44:58.344593 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:44:58.406970 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:58.406515 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" podUID="082c0bec-9706-4234-8ce4-e57f08aa23f5" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:44:59.065997 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:59.063927 4859 patch_prober.go:29] interesting pod/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:44:59.065997 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:59.063991 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" podUID="650965ab-b041-4c27-ab58-45c70ec37f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Nov 04 05:44:59.339123 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:59.339025 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" podStartSLOduration=5.339011302 podStartE2EDuration="5.339011302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:55.984718807 +0000 UTC m=+1065.986354554" watchObservedRunningTime="2026-11-04 05:44:59.339011302 +0000 UTC m=+1069.340647026" Nov 04 05:44:59.341363 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:59.341340 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm"] Nov 04 05:44:59.341699 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:44:59.341675 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" podUID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" containerName="istio-proxy" containerID="cri-o://ad34f79a1902e0a1110c8eb8d44901e0b3edb9e515922a90c2ca954205e214a8" gracePeriod=30 Nov 04 05:45:00.063480 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:00.063445 4859 patch_prober.go:29] interesting pod/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:00.063657 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:00.063498 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" podUID="650965ab-b041-4c27-ab58-45c70ec37f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Nov 04 05:45:01.062637 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:01.062605 4859 patch_prober.go:29] interesting pod/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:01.063127 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:01.062674 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" podUID="650965ab-b041-4c27-ab58-45c70ec37f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Nov 04 05:45:01.723782 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:01.723752 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm"] Nov 04 05:45:01.724006 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:01.723980 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" podUID="c684e324-0d8e-438d-b589-8a671f7c8b18" containerName="istio-proxy" containerID="cri-o://76825840171dd5bd48603ac811cedb4af5e01091aec7294fce5b8dfb05a76b4e" gracePeriod=30 Nov 04 05:45:01.911074 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:01.911039 4859 generic.go:356] "Generic (PLEG): container finished" podID="27f04fa4-41f7-4988-b5a4-d81187643cf7" containerID="1cc40cae53d8feb15552a5ed5847eda217a1c90c06c65d752cd5a135b111a7d5" exitCode=0 Nov 04 05:45:01.911185 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:01.911117 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" event={"ID":"27f04fa4-41f7-4988-b5a4-d81187643cf7","Type":"ContainerDied","Data":"1cc40cae53d8feb15552a5ed5847eda217a1c90c06c65d752cd5a135b111a7d5"} Nov 04 05:45:01.911237 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:01.911184 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" event={"ID":"27f04fa4-41f7-4988-b5a4-d81187643cf7","Type":"ContainerDied","Data":"e93094368c6d95bdecbc9cdbe0d93ae5a25d010a4d398571319c60dbdacea941"} Nov 04 05:45:01.911237 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:01.911211 4859 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e93094368c6d95bdecbc9cdbe0d93ae5a25d010a4d398571319c60dbdacea941" Nov 04 05:45:01.926602 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:01.926578 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" Nov 04 05:45:02.063176 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:02.063145 4859 patch_prober.go:29] interesting pod/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:02.063654 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:02.063193 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" podUID="650965ab-b041-4c27-ab58-45c70ec37f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Nov 04 05:45:02.075613 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:02.075580 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27f04fa4-41f7-4988-b5a4-d81187643cf7-kube-api-access-8ltx7\" (UniqueName: \"kubernetes.io/projected/27f04fa4-41f7-4988-b5a4-d81187643cf7-kube-api-access-8ltx7\") pod \"27f04fa4-41f7-4988-b5a4-d81187643cf7\" (UID: \"27f04fa4-41f7-4988-b5a4-d81187643cf7\") " Nov 04 05:45:02.075745 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:02.075687 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27f04fa4-41f7-4988-b5a4-d81187643cf7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27f04fa4-41f7-4988-b5a4-d81187643cf7-plugins-config-volume\") pod \"27f04fa4-41f7-4988-b5a4-d81187643cf7\" (UID: \"27f04fa4-41f7-4988-b5a4-d81187643cf7\") " Nov 04 05:45:02.076051 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:02.076030 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27f04fa4-41f7-4988-b5a4-d81187643cf7-plugins-config-volume" pod "27f04fa4-41f7-4988-b5a4-d81187643cf7" (UID: "27f04fa4-41f7-4988-b5a4-d81187643cf7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:02.076260 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:02.076232 4859 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27f04fa4-41f7-4988-b5a4-d81187643cf7-plugins-config-volume\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:02.077937 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:02.077909 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27f04fa4-41f7-4988-b5a4-d81187643cf7-kube-api-access-8ltx7" pod "27f04fa4-41f7-4988-b5a4-d81187643cf7" (UID: "27f04fa4-41f7-4988-b5a4-d81187643cf7"). InnerVolumeSpecName "kube-api-access-8ltx7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:02.177763 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:02.177726 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ltx7\" (UniqueName: \"kubernetes.io/projected/27f04fa4-41f7-4988-b5a4-d81187643cf7-kube-api-access-8ltx7\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:02.918985 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:02.918950 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f" Nov 04 05:45:03.100161 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:03.100123 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f"] Nov 04 05:45:03.162649 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:03.162614 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-283815-77ccd5c59-dpv4f"] Nov 04 05:45:03.233278 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:03.233189 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:45:03.233550 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:03.233529 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:45:03.234540 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:03.234516 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:45:03.393306 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:03.393270 4859 patch_prober.go:29] interesting pod/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:45:03.393448 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:03.393329 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" podUID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:45:04.116194 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.116163 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27f04fa4-41f7-4988-b5a4-d81187643cf7" path="/var/lib/kubelet/pods/27f04fa4-41f7-4988-b5a4-d81187643cf7/volumes" Nov 04 05:45:04.599089 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.599067 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:45:04.606064 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.606043 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-token\" (UniqueName: \"kubernetes.io/projected/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-token\") pod \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " Nov 04 05:45:04.606184 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.606074 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-workload-socket\") pod \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " Nov 04 05:45:04.606184 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.606099 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-podinfo\") pod \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " Nov 04 05:45:04.606184 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.606149 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istiod-ca-cert\") pod \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " Nov 04 05:45:04.606307 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.606250 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-envoy\") pod \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " Nov 04 05:45:04.606359 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.606300 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-workload-socket" pod "6fda8c73-74bb-49ce-b03e-e5517015ee6f" (UID: "6fda8c73-74bb-49ce-b03e-e5517015ee6f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:04.606359 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.606315 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-credential-socket\") pod \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " Nov 04 05:45:04.606463 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.606358 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-data\") pod \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " Nov 04 05:45:04.606463 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.606393 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-workload-certs\") pod \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " Nov 04 05:45:04.606463 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.606446 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fda8c73-74bb-49ce-b03e-e5517015ee6f-kube-api-access-fwtjk\" (UniqueName: \"kubernetes.io/projected/6fda8c73-74bb-49ce-b03e-e5517015ee6f-kube-api-access-fwtjk\") pod \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\" (UID: \"6fda8c73-74bb-49ce-b03e-e5517015ee6f\") " Nov 04 05:45:04.606644 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.606621 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-credential-socket" pod "6fda8c73-74bb-49ce-b03e-e5517015ee6f" (UID: "6fda8c73-74bb-49ce-b03e-e5517015ee6f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:04.606644 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.606633 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istiod-ca-cert" pod "6fda8c73-74bb-49ce-b03e-e5517015ee6f" (UID: "6fda8c73-74bb-49ce-b03e-e5517015ee6f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:04.606832 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.606808 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-data" pod "6fda8c73-74bb-49ce-b03e-e5517015ee6f" (UID: "6fda8c73-74bb-49ce-b03e-e5517015ee6f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:04.606925 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.606887 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-workload-certs" pod "6fda8c73-74bb-49ce-b03e-e5517015ee6f" (UID: "6fda8c73-74bb-49ce-b03e-e5517015ee6f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:04.607192 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.607169 4859 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-credential-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:04.607361 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.607338 4859 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-data\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:04.607461 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.607364 4859 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-workload-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:04.607461 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.607379 4859 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-workload-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:04.607461 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.607392 4859 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istiod-ca-cert\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:04.608385 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.608362 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-token" pod "6fda8c73-74bb-49ce-b03e-e5517015ee6f" (UID: "6fda8c73-74bb-49ce-b03e-e5517015ee6f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:04.608451 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.608422 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-envoy" pod "6fda8c73-74bb-49ce-b03e-e5517015ee6f" (UID: "6fda8c73-74bb-49ce-b03e-e5517015ee6f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:04.608559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.608543 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fda8c73-74bb-49ce-b03e-e5517015ee6f-kube-api-access-fwtjk" pod "6fda8c73-74bb-49ce-b03e-e5517015ee6f" (UID: "6fda8c73-74bb-49ce-b03e-e5517015ee6f"). InnerVolumeSpecName "kube-api-access-fwtjk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:04.608878 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.608855 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-podinfo" pod "6fda8c73-74bb-49ce-b03e-e5517015ee6f" (UID: "6fda8c73-74bb-49ce-b03e-e5517015ee6f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:45:04.708858 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.708823 4859 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-envoy\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:04.709009 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.708862 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwtjk\" (UniqueName: \"kubernetes.io/projected/6fda8c73-74bb-49ce-b03e-e5517015ee6f-kube-api-access-fwtjk\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:04.709009 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.708879 4859 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:04.709009 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.708893 4859 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fda8c73-74bb-49ce-b03e-e5517015ee6f-istio-podinfo\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:04.931458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.931426 4859 generic.go:356] "Generic (PLEG): container finished" podID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" containerID="ad34f79a1902e0a1110c8eb8d44901e0b3edb9e515922a90c2ca954205e214a8" exitCode=0 Nov 04 05:45:04.931624 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.931479 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" Nov 04 05:45:04.931624 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.931510 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" event={"ID":"6fda8c73-74bb-49ce-b03e-e5517015ee6f","Type":"ContainerDied","Data":"ad34f79a1902e0a1110c8eb8d44901e0b3edb9e515922a90c2ca954205e214a8"} Nov 04 05:45:04.931624 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.931578 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm" event={"ID":"6fda8c73-74bb-49ce-b03e-e5517015ee6f","Type":"ContainerDied","Data":"e94abd934f264eed8241a021523f40968b6a74f69e3e09b8583c6b8c41a19b1c"} Nov 04 05:45:04.931624 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.931608 4859 scope.go:122] "RemoveContainer" containerID="ad34f79a1902e0a1110c8eb8d44901e0b3edb9e515922a90c2ca954205e214a8" Nov 04 05:45:04.941991 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.941975 4859 scope.go:122] "RemoveContainer" containerID="a8d30f9bcd6b72a9f1b3fbce18e5a4e71d37e61b5e446510c32d858d4628876b" Nov 04 05:45:04.953125 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.953104 4859 scope.go:122] "RemoveContainer" containerID="ad34f79a1902e0a1110c8eb8d44901e0b3edb9e515922a90c2ca954205e214a8" Nov 04 05:45:04.953637 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:45:04.953613 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad34f79a1902e0a1110c8eb8d44901e0b3edb9e515922a90c2ca954205e214a8\": container with ID starting with ad34f79a1902e0a1110c8eb8d44901e0b3edb9e515922a90c2ca954205e214a8 not found: ID does not exist" containerID="ad34f79a1902e0a1110c8eb8d44901e0b3edb9e515922a90c2ca954205e214a8" Nov 04 05:45:04.953762 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.953643 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad34f79a1902e0a1110c8eb8d44901e0b3edb9e515922a90c2ca954205e214a8"} err="failed to get container status \"ad34f79a1902e0a1110c8eb8d44901e0b3edb9e515922a90c2ca954205e214a8\": rpc error: code = NotFound desc = could not find container \"ad34f79a1902e0a1110c8eb8d44901e0b3edb9e515922a90c2ca954205e214a8\": container with ID starting with ad34f79a1902e0a1110c8eb8d44901e0b3edb9e515922a90c2ca954205e214a8 not found: ID does not exist" Nov 04 05:45:04.953762 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.953687 4859 scope.go:122] "RemoveContainer" containerID="a8d30f9bcd6b72a9f1b3fbce18e5a4e71d37e61b5e446510c32d858d4628876b" Nov 04 05:45:04.954007 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:45:04.953988 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8d30f9bcd6b72a9f1b3fbce18e5a4e71d37e61b5e446510c32d858d4628876b\": container with ID starting with a8d30f9bcd6b72a9f1b3fbce18e5a4e71d37e61b5e446510c32d858d4628876b not found: ID does not exist" containerID="a8d30f9bcd6b72a9f1b3fbce18e5a4e71d37e61b5e446510c32d858d4628876b" Nov 04 05:45:04.954065 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:04.954016 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8d30f9bcd6b72a9f1b3fbce18e5a4e71d37e61b5e446510c32d858d4628876b"} err="failed to get container status \"a8d30f9bcd6b72a9f1b3fbce18e5a4e71d37e61b5e446510c32d858d4628876b\": rpc error: code = NotFound desc = could not find container \"a8d30f9bcd6b72a9f1b3fbce18e5a4e71d37e61b5e446510c32d858d4628876b\": container with ID starting with a8d30f9bcd6b72a9f1b3fbce18e5a4e71d37e61b5e446510c32d858d4628876b not found: ID does not exist" Nov 04 05:45:05.109458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:05.109376 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm"] Nov 04 05:45:05.113363 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:05.113333 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-098178-openshift-default-6b946b56b5-6dxhm"] Nov 04 05:45:05.167998 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:05.167960 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px"] Nov 04 05:45:05.937680 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:05.937627 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" podUID="650965ab-b041-4c27-ab58-45c70ec37f94" containerName="istio-proxy" containerID="cri-o://bd8ca6e751832d138d18b14304807bc375881cecc6cc9e6e33a4d79f5b09385c" gracePeriod=30 Nov 04 05:45:06.133521 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:06.133487 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" path="/var/lib/kubelet/pods/6fda8c73-74bb-49ce-b03e-e5517015ee6f/volumes" Nov 04 05:45:06.947630 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:06.947590 4859 generic.go:356] "Generic (PLEG): container finished" podID="c684e324-0d8e-438d-b589-8a671f7c8b18" containerID="76825840171dd5bd48603ac811cedb4af5e01091aec7294fce5b8dfb05a76b4e" exitCode=0 Nov 04 05:45:06.948113 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:06.947643 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" event={"ID":"c684e324-0d8e-438d-b589-8a671f7c8b18","Type":"ContainerDied","Data":"76825840171dd5bd48603ac811cedb4af5e01091aec7294fce5b8dfb05a76b4e"} Nov 04 05:45:06.948113 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:06.947759 4859 scope.go:122] "RemoveContainer" containerID="513229f772e0aad21292d0ad06222275823360bb94f81f2952b71caea3516569" Nov 04 05:45:06.996609 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:06.996562 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:45:07.039885 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.039853 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-workload-certs\") pod \"c684e324-0d8e-438d-b589-8a671f7c8b18\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " Nov 04 05:45:07.039984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.039895 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-token\" (UniqueName: \"kubernetes.io/projected/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-token\") pod \"c684e324-0d8e-438d-b589-8a671f7c8b18\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " Nov 04 05:45:07.039984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.039915 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-workload-socket\") pod \"c684e324-0d8e-438d-b589-8a671f7c8b18\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " Nov 04 05:45:07.039984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.039950 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-envoy\") pod \"c684e324-0d8e-438d-b589-8a671f7c8b18\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " Nov 04 05:45:07.039984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.039968 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c684e324-0d8e-438d-b589-8a671f7c8b18-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c684e324-0d8e-438d-b589-8a671f7c8b18-istiod-ca-cert\") pod \"c684e324-0d8e-438d-b589-8a671f7c8b18\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " Nov 04 05:45:07.040129 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.040054 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c684e324-0d8e-438d-b589-8a671f7c8b18-kube-api-access-vsrt4\" (UniqueName: \"kubernetes.io/projected/c684e324-0d8e-438d-b589-8a671f7c8b18-kube-api-access-vsrt4\") pod \"c684e324-0d8e-438d-b589-8a671f7c8b18\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " Nov 04 05:45:07.040129 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.040105 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-podinfo\") pod \"c684e324-0d8e-438d-b589-8a671f7c8b18\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " Nov 04 05:45:07.040129 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.040116 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-workload-certs" pod "c684e324-0d8e-438d-b589-8a671f7c8b18" (UID: "c684e324-0d8e-438d-b589-8a671f7c8b18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:07.040292 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.040147 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-data\") pod \"c684e324-0d8e-438d-b589-8a671f7c8b18\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " Nov 04 05:45:07.040292 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.040187 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-credential-socket\") pod \"c684e324-0d8e-438d-b589-8a671f7c8b18\" (UID: \"c684e324-0d8e-438d-b589-8a671f7c8b18\") " Nov 04 05:45:07.040415 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.040340 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-workload-socket" pod "c684e324-0d8e-438d-b589-8a671f7c8b18" (UID: "c684e324-0d8e-438d-b589-8a671f7c8b18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:07.040415 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.040364 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-data" pod "c684e324-0d8e-438d-b589-8a671f7c8b18" (UID: "c684e324-0d8e-438d-b589-8a671f7c8b18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:07.040569 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.040542 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-credential-socket" pod "c684e324-0d8e-438d-b589-8a671f7c8b18" (UID: "c684e324-0d8e-438d-b589-8a671f7c8b18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:07.040651 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.040583 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c684e324-0d8e-438d-b589-8a671f7c8b18-istiod-ca-cert" pod "c684e324-0d8e-438d-b589-8a671f7c8b18" (UID: "c684e324-0d8e-438d-b589-8a671f7c8b18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:07.040794 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.040769 4859 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-workload-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.040794 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.040797 4859 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-workload-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.040969 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.040807 4859 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c684e324-0d8e-438d-b589-8a671f7c8b18-istiod-ca-cert\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.040969 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.040815 4859 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-data\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.040969 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.040825 4859 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-credential-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.042181 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.042153 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-token" pod "c684e324-0d8e-438d-b589-8a671f7c8b18" (UID: "c684e324-0d8e-438d-b589-8a671f7c8b18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:07.042382 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.042357 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c684e324-0d8e-438d-b589-8a671f7c8b18-kube-api-access-vsrt4" pod "c684e324-0d8e-438d-b589-8a671f7c8b18" (UID: "c684e324-0d8e-438d-b589-8a671f7c8b18"). InnerVolumeSpecName "kube-api-access-vsrt4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:07.042625 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.042602 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-envoy" pod "c684e324-0d8e-438d-b589-8a671f7c8b18" (UID: "c684e324-0d8e-438d-b589-8a671f7c8b18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:07.042736 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.042687 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-podinfo" pod "c684e324-0d8e-438d-b589-8a671f7c8b18" (UID: "c684e324-0d8e-438d-b589-8a671f7c8b18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:45:07.142162 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.142129 4859 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-podinfo\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.142162 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.142158 4859 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.142162 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.142167 4859 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c684e324-0d8e-438d-b589-8a671f7c8b18-istio-envoy\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.142345 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.142176 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsrt4\" (UniqueName: \"kubernetes.io/projected/c684e324-0d8e-438d-b589-8a671f7c8b18-kube-api-access-vsrt4\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.955615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.955587 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" Nov 04 05:45:07.955615 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.955609 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm" event={"ID":"c684e324-0d8e-438d-b589-8a671f7c8b18","Type":"ContainerDied","Data":"d94b1c4c8820ae6d52661b019e3e93d4d6b66c34da5e07997a1df07bcd2ab0c4"} Nov 04 05:45:07.956101 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:07.955660 4859 scope.go:122] "RemoveContainer" containerID="76825840171dd5bd48603ac811cedb4af5e01091aec7294fce5b8dfb05a76b4e" Nov 04 05:45:08.187718 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:08.187691 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm"] Nov 04 05:45:08.241970 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:08.241909 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-l2zsm"] Nov 04 05:45:08.385229 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:08.385198 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" podUID="082c0bec-9706-4234-8ce4-e57f08aa23f5" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:45:09.318190 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:09.318130 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" Nov 04 05:45:09.365995 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:09.365967 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/082c0bec-9706-4234-8ce4-e57f08aa23f5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/082c0bec-9706-4234-8ce4-e57f08aa23f5-plugins-config-volume\") pod \"082c0bec-9706-4234-8ce4-e57f08aa23f5\" (UID: \"082c0bec-9706-4234-8ce4-e57f08aa23f5\") " Nov 04 05:45:09.366128 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:09.366023 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/082c0bec-9706-4234-8ce4-e57f08aa23f5-kube-api-access-krgc6\" (UniqueName: \"kubernetes.io/projected/082c0bec-9706-4234-8ce4-e57f08aa23f5-kube-api-access-krgc6\") pod \"082c0bec-9706-4234-8ce4-e57f08aa23f5\" (UID: \"082c0bec-9706-4234-8ce4-e57f08aa23f5\") " Nov 04 05:45:09.366317 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:09.366294 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/082c0bec-9706-4234-8ce4-e57f08aa23f5-plugins-config-volume" pod "082c0bec-9706-4234-8ce4-e57f08aa23f5" (UID: "082c0bec-9706-4234-8ce4-e57f08aa23f5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:09.366483 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:09.366468 4859 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/082c0bec-9706-4234-8ce4-e57f08aa23f5-plugins-config-volume\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:09.367935 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:09.367917 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/082c0bec-9706-4234-8ce4-e57f08aa23f5-kube-api-access-krgc6" pod "082c0bec-9706-4234-8ce4-e57f08aa23f5" (UID: "082c0bec-9706-4234-8ce4-e57f08aa23f5"). InnerVolumeSpecName "kube-api-access-krgc6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:09.467750 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:09.467726 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krgc6\" (UniqueName: \"kubernetes.io/projected/082c0bec-9706-4234-8ce4-e57f08aa23f5-kube-api-access-krgc6\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:09.968657 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:09.968618 4859 generic.go:356] "Generic (PLEG): container finished" podID="082c0bec-9706-4234-8ce4-e57f08aa23f5" containerID="13d852f8e60388b7dc548e556681d6fa5d09f80bf28c08f82594b6afad1b3b8e" exitCode=0 Nov 04 05:45:09.968868 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:09.968682 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" Nov 04 05:45:09.968868 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:09.968703 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" event={"ID":"082c0bec-9706-4234-8ce4-e57f08aa23f5","Type":"ContainerDied","Data":"13d852f8e60388b7dc548e556681d6fa5d09f80bf28c08f82594b6afad1b3b8e"} Nov 04 05:45:09.968868 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:09.968774 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj" event={"ID":"082c0bec-9706-4234-8ce4-e57f08aa23f5","Type":"ContainerDied","Data":"ec4b403540265404762a1bf0f8ec8372e6275447fa88298a6d4078099da38c54"} Nov 04 05:45:09.968868 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:09.968808 4859 scope.go:122] "RemoveContainer" containerID="13d852f8e60388b7dc548e556681d6fa5d09f80bf28c08f82594b6afad1b3b8e" Nov 04 05:45:09.977983 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:09.977968 4859 scope.go:122] "RemoveContainer" containerID="13d852f8e60388b7dc548e556681d6fa5d09f80bf28c08f82594b6afad1b3b8e" Nov 04 05:45:09.978231 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:45:09.978213 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13d852f8e60388b7dc548e556681d6fa5d09f80bf28c08f82594b6afad1b3b8e\": container with ID starting with 13d852f8e60388b7dc548e556681d6fa5d09f80bf28c08f82594b6afad1b3b8e not found: ID does not exist" containerID="13d852f8e60388b7dc548e556681d6fa5d09f80bf28c08f82594b6afad1b3b8e" Nov 04 05:45:09.978269 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:09.978240 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13d852f8e60388b7dc548e556681d6fa5d09f80bf28c08f82594b6afad1b3b8e"} err="failed to get container status \"13d852f8e60388b7dc548e556681d6fa5d09f80bf28c08f82594b6afad1b3b8e\": rpc error: code = NotFound desc = could not find container \"13d852f8e60388b7dc548e556681d6fa5d09f80bf28c08f82594b6afad1b3b8e\": container with ID starting with 13d852f8e60388b7dc548e556681d6fa5d09f80bf28c08f82594b6afad1b3b8e not found: ID does not exist" Nov 04 05:45:10.116774 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:10.116746 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c684e324-0d8e-438d-b589-8a671f7c8b18" path="/var/lib/kubelet/pods/c684e324-0d8e-438d-b589-8a671f7c8b18/volumes" Nov 04 05:45:10.146799 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:10.146774 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj"] Nov 04 05:45:10.201207 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:10.201184 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-fbbad9-5cbd955cbc-7bdcj"] Nov 04 05:45:11.195783 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.195755 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:45:11.286233 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.286154 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-istio-data\") pod \"650965ab-b041-4c27-ab58-45c70ec37f94\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " Nov 04 05:45:11.286233 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.286209 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-istio-envoy\") pod \"650965ab-b041-4c27-ab58-45c70ec37f94\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " Nov 04 05:45:11.286419 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.286375 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/650965ab-b041-4c27-ab58-45c70ec37f94-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/650965ab-b041-4c27-ab58-45c70ec37f94-istio-podinfo\") pod \"650965ab-b041-4c27-ab58-45c70ec37f94\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " Nov 04 05:45:11.286419 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.286408 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/650965ab-b041-4c27-ab58-45c70ec37f94-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/650965ab-b041-4c27-ab58-45c70ec37f94-istiod-ca-cert\") pod \"650965ab-b041-4c27-ab58-45c70ec37f94\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " Nov 04 05:45:11.286529 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.286450 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/650965ab-b041-4c27-ab58-45c70ec37f94-kube-api-access-svw7h\" (UniqueName: \"kubernetes.io/projected/650965ab-b041-4c27-ab58-45c70ec37f94-kube-api-access-svw7h\") pod \"650965ab-b041-4c27-ab58-45c70ec37f94\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " Nov 04 05:45:11.286529 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.286468 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-istio-data" pod "650965ab-b041-4c27-ab58-45c70ec37f94" (UID: "650965ab-b041-4c27-ab58-45c70ec37f94"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:11.286529 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.286483 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-credential-socket\") pod \"650965ab-b041-4c27-ab58-45c70ec37f94\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " Nov 04 05:45:11.286701 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.286580 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-workload-socket\") pod \"650965ab-b041-4c27-ab58-45c70ec37f94\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " Nov 04 05:45:11.286771 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.286715 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-workload-certs\") pod \"650965ab-b041-4c27-ab58-45c70ec37f94\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " Nov 04 05:45:11.286831 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.286769 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/650965ab-b041-4c27-ab58-45c70ec37f94-istio-token\" (UniqueName: \"kubernetes.io/projected/650965ab-b041-4c27-ab58-45c70ec37f94-istio-token\") pod \"650965ab-b041-4c27-ab58-45c70ec37f94\" (UID: \"650965ab-b041-4c27-ab58-45c70ec37f94\") " Nov 04 05:45:11.286831 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.286771 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-credential-socket" pod "650965ab-b041-4c27-ab58-45c70ec37f94" (UID: "650965ab-b041-4c27-ab58-45c70ec37f94"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:11.286928 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.286885 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-workload-socket" pod "650965ab-b041-4c27-ab58-45c70ec37f94" (UID: "650965ab-b041-4c27-ab58-45c70ec37f94"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:11.286980 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.286890 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/650965ab-b041-4c27-ab58-45c70ec37f94-istiod-ca-cert" pod "650965ab-b041-4c27-ab58-45c70ec37f94" (UID: "650965ab-b041-4c27-ab58-45c70ec37f94"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:11.287026 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.286985 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-workload-certs" pod "650965ab-b041-4c27-ab58-45c70ec37f94" (UID: "650965ab-b041-4c27-ab58-45c70ec37f94"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:11.287252 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.287220 4859 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-istio-data\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:11.287307 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.287258 4859 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/650965ab-b041-4c27-ab58-45c70ec37f94-istiod-ca-cert\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:11.287307 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.287274 4859 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-credential-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:11.287307 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.287287 4859 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-workload-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:11.287307 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.287304 4859 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-workload-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:11.288634 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.288601 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-istio-envoy" pod "650965ab-b041-4c27-ab58-45c70ec37f94" (UID: "650965ab-b041-4c27-ab58-45c70ec37f94"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:11.288775 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.288730 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/650965ab-b041-4c27-ab58-45c70ec37f94-kube-api-access-svw7h" pod "650965ab-b041-4c27-ab58-45c70ec37f94" (UID: "650965ab-b041-4c27-ab58-45c70ec37f94"). InnerVolumeSpecName "kube-api-access-svw7h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:11.288936 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.288920 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/650965ab-b041-4c27-ab58-45c70ec37f94-istio-podinfo" pod "650965ab-b041-4c27-ab58-45c70ec37f94" (UID: "650965ab-b041-4c27-ab58-45c70ec37f94"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:45:11.289279 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.289266 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/650965ab-b041-4c27-ab58-45c70ec37f94-istio-token" pod "650965ab-b041-4c27-ab58-45c70ec37f94" (UID: "650965ab-b041-4c27-ab58-45c70ec37f94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:11.389258 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.389223 4859 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/650965ab-b041-4c27-ab58-45c70ec37f94-istio-envoy\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:11.389258 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.389258 4859 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/650965ab-b041-4c27-ab58-45c70ec37f94-istio-podinfo\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:11.389446 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.389273 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svw7h\" (UniqueName: \"kubernetes.io/projected/650965ab-b041-4c27-ab58-45c70ec37f94-kube-api-access-svw7h\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:11.389446 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.389288 4859 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/650965ab-b041-4c27-ab58-45c70ec37f94-istio-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:11.980626 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.980590 4859 generic.go:356] "Generic (PLEG): container finished" podID="650965ab-b041-4c27-ab58-45c70ec37f94" containerID="bd8ca6e751832d138d18b14304807bc375881cecc6cc9e6e33a4d79f5b09385c" exitCode=0 Nov 04 05:45:11.980818 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.980653 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" event={"ID":"650965ab-b041-4c27-ab58-45c70ec37f94","Type":"ContainerDied","Data":"bd8ca6e751832d138d18b14304807bc375881cecc6cc9e6e33a4d79f5b09385c"} Nov 04 05:45:11.980818 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.980744 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" event={"ID":"650965ab-b041-4c27-ab58-45c70ec37f94","Type":"ContainerDied","Data":"65b9e5d8da52faf44e8fbecbe2dd500ac6a888f39a0bccadefd21e50bd1b98f4"} Nov 04 05:45:11.980818 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.980652 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px" Nov 04 05:45:11.980818 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.980767 4859 scope.go:122] "RemoveContainer" containerID="bd8ca6e751832d138d18b14304807bc375881cecc6cc9e6e33a4d79f5b09385c" Nov 04 05:45:11.990256 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.990235 4859 scope.go:122] "RemoveContainer" containerID="bd8ca6e751832d138d18b14304807bc375881cecc6cc9e6e33a4d79f5b09385c" Nov 04 05:45:11.990521 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:45:11.990500 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd8ca6e751832d138d18b14304807bc375881cecc6cc9e6e33a4d79f5b09385c\": container with ID starting with bd8ca6e751832d138d18b14304807bc375881cecc6cc9e6e33a4d79f5b09385c not found: ID does not exist" containerID="bd8ca6e751832d138d18b14304807bc375881cecc6cc9e6e33a4d79f5b09385c" Nov 04 05:45:11.990572 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:11.990529 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd8ca6e751832d138d18b14304807bc375881cecc6cc9e6e33a4d79f5b09385c"} err="failed to get container status \"bd8ca6e751832d138d18b14304807bc375881cecc6cc9e6e33a4d79f5b09385c\": rpc error: code = NotFound desc = could not find container \"bd8ca6e751832d138d18b14304807bc375881cecc6cc9e6e33a4d79f5b09385c\": container with ID starting with bd8ca6e751832d138d18b14304807bc375881cecc6cc9e6e33a4d79f5b09385c not found: ID does not exist" Nov 04 05:45:12.115629 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:12.115602 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="082c0bec-9706-4234-8ce4-e57f08aa23f5" path="/var/lib/kubelet/pods/082c0bec-9706-4234-8ce4-e57f08aa23f5/volumes" Nov 04 05:45:12.161234 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:12.161195 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px"] Nov 04 05:45:12.214776 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:12.214749 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-kx8px"] Nov 04 05:45:12.988724 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:12.988692 4859 generic.go:356] "Generic (PLEG): container finished" podID="768e34fb-e3f3-4b22-9012-1c8f11b38b76" containerID="1612d4e70cb6d164e1a47fc5c6b90f6cc12f00b5045024a51a8b378bb7bc1b29" exitCode=0 Nov 04 05:45:12.988837 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:12.988740 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rjx94-bzck8" event={"ID":"768e34fb-e3f3-4b22-9012-1c8f11b38b76","Type":"ContainerDied","Data":"1612d4e70cb6d164e1a47fc5c6b90f6cc12f00b5045024a51a8b378bb7bc1b29"} Nov 04 05:45:14.115529 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:14.115495 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="650965ab-b041-4c27-ab58-45c70ec37f94" path="/var/lib/kubelet/pods/650965ab-b041-4c27-ab58-45c70ec37f94/volumes" Nov 04 05:45:14.145118 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:14.145095 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rjx94-bzck8" Nov 04 05:45:14.217113 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:14.217088 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/768e34fb-e3f3-4b22-9012-1c8f11b38b76-kube-api-access-f5n2q\" (UniqueName: \"kubernetes.io/projected/768e34fb-e3f3-4b22-9012-1c8f11b38b76-kube-api-access-f5n2q\") pod \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\" (UID: \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\") " Nov 04 05:45:14.217252 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:14.217141 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/768e34fb-e3f3-4b22-9012-1c8f11b38b76-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768e34fb-e3f3-4b22-9012-1c8f11b38b76-service-ca-bundle\") pod \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\" (UID: \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\") " Nov 04 05:45:14.217252 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:14.217191 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/768e34fb-e3f3-4b22-9012-1c8f11b38b76-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768e34fb-e3f3-4b22-9012-1c8f11b38b76-trusted-ca-bundle\") pod \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\" (UID: \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\") " Nov 04 05:45:14.217252 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:14.217248 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/768e34fb-e3f3-4b22-9012-1c8f11b38b76-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/768e34fb-e3f3-4b22-9012-1c8f11b38b76-archives-path\") pod \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\" (UID: \"768e34fb-e3f3-4b22-9012-1c8f11b38b76\") " Nov 04 05:45:14.217624 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:14.217584 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/768e34fb-e3f3-4b22-9012-1c8f11b38b76-trusted-ca-bundle" pod "768e34fb-e3f3-4b22-9012-1c8f11b38b76" (UID: "768e34fb-e3f3-4b22-9012-1c8f11b38b76"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:14.217624 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:14.217594 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/768e34fb-e3f3-4b22-9012-1c8f11b38b76-service-ca-bundle" pod "768e34fb-e3f3-4b22-9012-1c8f11b38b76" (UID: "768e34fb-e3f3-4b22-9012-1c8f11b38b76"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:14.217812 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:14.217642 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/768e34fb-e3f3-4b22-9012-1c8f11b38b76-archives-path" pod "768e34fb-e3f3-4b22-9012-1c8f11b38b76" (UID: "768e34fb-e3f3-4b22-9012-1c8f11b38b76"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:14.219395 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:14.219367 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/768e34fb-e3f3-4b22-9012-1c8f11b38b76-kube-api-access-f5n2q" pod "768e34fb-e3f3-4b22-9012-1c8f11b38b76" (UID: "768e34fb-e3f3-4b22-9012-1c8f11b38b76"). InnerVolumeSpecName "kube-api-access-f5n2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:14.318761 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:14.318698 4859 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768e34fb-e3f3-4b22-9012-1c8f11b38b76-service-ca-bundle\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:14.318761 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:14.318721 4859 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768e34fb-e3f3-4b22-9012-1c8f11b38b76-trusted-ca-bundle\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:14.318761 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:14.318731 4859 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/768e34fb-e3f3-4b22-9012-1c8f11b38b76-archives-path\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:14.318761 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:14.318740 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5n2q\" (UniqueName: \"kubernetes.io/projected/768e34fb-e3f3-4b22-9012-1c8f11b38b76-kube-api-access-f5n2q\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:15.001281 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:15.001250 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rjx94-bzck8" event={"ID":"768e34fb-e3f3-4b22-9012-1c8f11b38b76","Type":"ContainerDied","Data":"13ecc4d82d3985ebbba26cec9d2b05d6477055eaf42a6a12db904d7c18559dfa"} Nov 04 05:45:15.001281 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:15.001284 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rjx94-bzck8" Nov 04 05:45:15.001529 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:15.001294 4859 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="13ecc4d82d3985ebbba26cec9d2b05d6477055eaf42a6a12db904d7c18559dfa" Nov 04 05:45:28.342313 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:45:28.342280 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:45:30.820128 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:30.820098 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4"] Nov 04 05:45:38.980020 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:38.979986 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746"] Nov 04 05:45:39.106252 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:39.106215 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l"] Nov 04 05:45:43.980661 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:43.980603 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" podUID="3567dc66-3594-4163-a4d1-e2a4e9d5b408" containerName="payload-processing" containerID="cri-o://3db1fa2a750118eceabe9cc94366cd6d10cc21732de992558a3b07c2f4b257b6" gracePeriod=25 Nov 04 05:45:44.107410 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:44.107370 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" podUID="74964f80-4702-460f-95bd-864bd6b30d93" containerName="payload-pre-processing" containerID="cri-o://0408db1f14fccd416dd3e51d7646370b1dd402c89da0abec345839d03078b8de" gracePeriod=25 Nov 04 05:45:44.441571 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:44.441529 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" podUID="74964f80-4702-460f-95bd-864bd6b30d93" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:45:44.441793 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:44.441600 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" podUID="3567dc66-3594-4163-a4d1-e2a4e9d5b408" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:45:45.840199 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:45.840151 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" podUID="6283c937-1f92-49c6-a17b-7cf069384263" containerName="main" containerID="cri-o://7147a2b5cd95210ee0dce5b6363b903a3a2d01741e0d71c475a22f308f1d7dc4" gracePeriod=45 Nov 04 05:45:46.105569 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.105510 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:45:46.128977 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.128949 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6283c937-1f92-49c6-a17b-7cf069384263-kube-api-access-rnh4r\" (UniqueName: \"kubernetes.io/projected/6283c937-1f92-49c6-a17b-7cf069384263-kube-api-access-rnh4r\") pod \"6283c937-1f92-49c6-a17b-7cf069384263\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " Nov 04 05:45:46.129114 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.128994 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6283c937-1f92-49c6-a17b-7cf069384263-tls-certs\" (UniqueName: \"kubernetes.io/secret/6283c937-1f92-49c6-a17b-7cf069384263-tls-certs\") pod \"6283c937-1f92-49c6-a17b-7cf069384263\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " Nov 04 05:45:46.129184 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.129109 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-home\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-home\") pod \"6283c937-1f92-49c6-a17b-7cf069384263\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " Nov 04 05:45:46.129245 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.129223 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-dshm\") pod \"6283c937-1f92-49c6-a17b-7cf069384263\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " Nov 04 05:45:46.129306 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.129272 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-tmp-dir\") pod \"6283c937-1f92-49c6-a17b-7cf069384263\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " Nov 04 05:45:46.129362 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.129326 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-model-cache\") pod \"6283c937-1f92-49c6-a17b-7cf069384263\" (UID: \"6283c937-1f92-49c6-a17b-7cf069384263\") " Nov 04 05:45:46.129362 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.129328 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-home" pod "6283c937-1f92-49c6-a17b-7cf069384263" (UID: "6283c937-1f92-49c6-a17b-7cf069384263"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:46.129594 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.129548 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-tmp-dir" pod "6283c937-1f92-49c6-a17b-7cf069384263" (UID: "6283c937-1f92-49c6-a17b-7cf069384263"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:46.129718 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.129601 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-model-cache" pod "6283c937-1f92-49c6-a17b-7cf069384263" (UID: "6283c937-1f92-49c6-a17b-7cf069384263"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:46.129842 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.129824 4859 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-tmp-dir\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.129886 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.129849 4859 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-model-cache\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.129886 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.129863 4859 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-home\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.131130 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.131100 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6283c937-1f92-49c6-a17b-7cf069384263-kube-api-access-rnh4r" pod "6283c937-1f92-49c6-a17b-7cf069384263" (UID: "6283c937-1f92-49c6-a17b-7cf069384263"). InnerVolumeSpecName "kube-api-access-rnh4r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:46.131130 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.131119 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6283c937-1f92-49c6-a17b-7cf069384263-tls-certs" pod "6283c937-1f92-49c6-a17b-7cf069384263" (UID: "6283c937-1f92-49c6-a17b-7cf069384263"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:46.131251 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.131158 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-dshm" pod "6283c937-1f92-49c6-a17b-7cf069384263" (UID: "6283c937-1f92-49c6-a17b-7cf069384263"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:46.167953 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.167927 4859 generic.go:356] "Generic (PLEG): container finished" podID="6283c937-1f92-49c6-a17b-7cf069384263" containerID="7147a2b5cd95210ee0dce5b6363b903a3a2d01741e0d71c475a22f308f1d7dc4" exitCode=0 Nov 04 05:45:46.168063 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.167967 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" Nov 04 05:45:46.168063 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.168008 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" event={"ID":"6283c937-1f92-49c6-a17b-7cf069384263","Type":"ContainerDied","Data":"7147a2b5cd95210ee0dce5b6363b903a3a2d01741e0d71c475a22f308f1d7dc4"} Nov 04 05:45:46.168063 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.168053 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4" event={"ID":"6283c937-1f92-49c6-a17b-7cf069384263","Type":"ContainerDied","Data":"8a6d085db00b459375d9eaa6c35f480c6ac2f9829e95cb78eade579a2a3edd45"} Nov 04 05:45:46.168224 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.168074 4859 scope.go:122] "RemoveContainer" containerID="7147a2b5cd95210ee0dce5b6363b903a3a2d01741e0d71c475a22f308f1d7dc4" Nov 04 05:45:46.177780 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.177761 4859 scope.go:122] "RemoveContainer" containerID="7147a2b5cd95210ee0dce5b6363b903a3a2d01741e0d71c475a22f308f1d7dc4" Nov 04 05:45:46.178020 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:45:46.178002 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7147a2b5cd95210ee0dce5b6363b903a3a2d01741e0d71c475a22f308f1d7dc4\": container with ID starting with 7147a2b5cd95210ee0dce5b6363b903a3a2d01741e0d71c475a22f308f1d7dc4 not found: ID does not exist" containerID="7147a2b5cd95210ee0dce5b6363b903a3a2d01741e0d71c475a22f308f1d7dc4" Nov 04 05:45:46.178075 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.178027 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7147a2b5cd95210ee0dce5b6363b903a3a2d01741e0d71c475a22f308f1d7dc4"} err="failed to get container status \"7147a2b5cd95210ee0dce5b6363b903a3a2d01741e0d71c475a22f308f1d7dc4\": rpc error: code = NotFound desc = could not find container \"7147a2b5cd95210ee0dce5b6363b903a3a2d01741e0d71c475a22f308f1d7dc4\": container with ID starting with 7147a2b5cd95210ee0dce5b6363b903a3a2d01741e0d71c475a22f308f1d7dc4 not found: ID does not exist" Nov 04 05:45:46.248716 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.236606 4859 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6283c937-1f92-49c6-a17b-7cf069384263-dshm\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.248716 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.236641 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnh4r\" (UniqueName: \"kubernetes.io/projected/6283c937-1f92-49c6-a17b-7cf069384263-kube-api-access-rnh4r\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.248716 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.236655 4859 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6283c937-1f92-49c6-a17b-7cf069384263-tls-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.358360 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.358283 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4"] Nov 04 05:45:46.361912 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:46.361880 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/distinct-w3-a84d48-kserve-79dbd88975-zr6l4"] Nov 04 05:45:48.115070 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:48.115037 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6283c937-1f92-49c6-a17b-7cf069384263" path="/var/lib/kubelet/pods/6283c937-1f92-49c6-a17b-7cf069384263/volumes" Nov 04 05:45:54.441495 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.441406 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" podUID="74964f80-4702-460f-95bd-864bd6b30d93" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:45:54.441495 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.441433 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" podUID="3567dc66-3594-4163-a4d1-e2a4e9d5b408" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:45:54.912815 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.912741 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9"] Nov 04 05:45:54.913241 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913226 4859 cpu_manager.go:414] "containerMap: removing container" podUID="27f04fa4-41f7-4988-b5a4-d81187643cf7" containerName="payload-pre-processing" Nov 04 05:45:54.913285 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913243 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27f04fa4-41f7-4988-b5a4-d81187643cf7" containerName="payload-pre-processing" Nov 04 05:45:54.913285 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913256 4859 cpu_manager.go:414] "containerMap: removing container" podUID="768e34fb-e3f3-4b22-9012-1c8f11b38b76" containerName="insights-gathering" Nov 04 05:45:54.913285 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913261 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="768e34fb-e3f3-4b22-9012-1c8f11b38b76" containerName="insights-gathering" Nov 04 05:45:54.913285 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913272 4859 cpu_manager.go:414] "containerMap: removing container" podUID="082c0bec-9706-4234-8ce4-e57f08aa23f5" containerName="payload-pre-processing" Nov 04 05:45:54.913285 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913277 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="082c0bec-9706-4234-8ce4-e57f08aa23f5" containerName="payload-pre-processing" Nov 04 05:45:54.913434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913295 4859 cpu_manager.go:414] "containerMap: removing container" podUID="c684e324-0d8e-438d-b589-8a671f7c8b18" containerName="istio-proxy" Nov 04 05:45:54.913434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913300 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c684e324-0d8e-438d-b589-8a671f7c8b18" containerName="istio-proxy" Nov 04 05:45:54.913434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913308 4859 cpu_manager.go:414] "containerMap: removing container" podUID="6283c937-1f92-49c6-a17b-7cf069384263" containerName="main" Nov 04 05:45:54.913434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913312 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6283c937-1f92-49c6-a17b-7cf069384263" containerName="main" Nov 04 05:45:54.913434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913319 4859 cpu_manager.go:414] "containerMap: removing container" podUID="650965ab-b041-4c27-ab58-45c70ec37f94" containerName="istio-proxy" Nov 04 05:45:54.913434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913323 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="650965ab-b041-4c27-ab58-45c70ec37f94" containerName="istio-proxy" Nov 04 05:45:54.913434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913335 4859 cpu_manager.go:414] "containerMap: removing container" podUID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" containerName="istio-proxy" Nov 04 05:45:54.913434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913339 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" containerName="istio-proxy" Nov 04 05:45:54.913434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913345 4859 cpu_manager.go:414] "containerMap: removing container" podUID="c684e324-0d8e-438d-b589-8a671f7c8b18" containerName="istio-proxy" Nov 04 05:45:54.913434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913349 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c684e324-0d8e-438d-b589-8a671f7c8b18" containerName="istio-proxy" Nov 04 05:45:54.913434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913361 4859 cpu_manager.go:414] "containerMap: removing container" podUID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" containerName="istio-proxy" Nov 04 05:45:54.913434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913365 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" containerName="istio-proxy" Nov 04 05:45:54.913434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913426 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" containerName="istio-proxy" Nov 04 05:45:54.913434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913434 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="27f04fa4-41f7-4988-b5a4-d81187643cf7" containerName="payload-pre-processing" Nov 04 05:45:54.913434 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913441 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="768e34fb-e3f3-4b22-9012-1c8f11b38b76" containerName="insights-gathering" Nov 04 05:45:54.913878 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913448 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="082c0bec-9706-4234-8ce4-e57f08aa23f5" containerName="payload-pre-processing" Nov 04 05:45:54.913878 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913455 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="c684e324-0d8e-438d-b589-8a671f7c8b18" containerName="istio-proxy" Nov 04 05:45:54.913878 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913461 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="6283c937-1f92-49c6-a17b-7cf069384263" containerName="main" Nov 04 05:45:54.913878 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913467 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="650965ab-b041-4c27-ab58-45c70ec37f94" containerName="istio-proxy" Nov 04 05:45:54.913878 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.913473 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="c684e324-0d8e-438d-b589-8a671f7c8b18" containerName="istio-proxy" Nov 04 05:45:54.917954 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:54.917937 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.021570 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.021537 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp6xr\" (UniqueName: \"kubernetes.io/projected/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-kube-api-access-pp6xr\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.021570 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.021577 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-workload-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.021787 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.021601 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-credential-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.021787 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.021657 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-workload-certs\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.021787 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.021760 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-token\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.021888 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.021796 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-podinfo\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.021888 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.021870 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-envoy\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.021950 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.021916 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istiod-ca-cert\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.021984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.021946 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-data\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.031961 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.031932 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9"] Nov 04 05:45:55.123284 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.123250 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-data\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.123447 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.123345 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp6xr\" (UniqueName: \"kubernetes.io/projected/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-kube-api-access-pp6xr\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.123447 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.123371 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-workload-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.123447 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.123401 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-credential-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.123447 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.123424 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-workload-certs\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.123620 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.123465 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-token\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.123620 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.123509 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-podinfo\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.123620 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.123583 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-envoy\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.123620 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.123613 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istiod-ca-cert\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.123860 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.123757 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-data\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.123903 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.123850 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-credential-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.123903 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.123875 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-workload-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.123903 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.123894 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-workload-certs\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.124302 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.124282 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istiod-ca-cert\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.125808 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.125785 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-envoy\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.126098 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.126083 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-podinfo\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.188532 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.188506 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-token\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.188695 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.188652 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp6xr\" (UniqueName: \"kubernetes.io/projected/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-kube-api-access-pp6xr\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.238816 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.238790 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:55.684945 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.684915 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9"] Nov 04 05:45:55.688885 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:45:55.688862 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2912c2d6_a4ad_4253_b9fa_f51c26afdd8d.slice/crio-a6c371ff99f9730d1ddf6620524bd7f4af51c8325689758f86541b865528ea7e WatchSource:0}: Error finding container a6c371ff99f9730d1ddf6620524bd7f4af51c8325689758f86541b865528ea7e: Status 404 returned error can't find the container with id a6c371ff99f9730d1ddf6620524bd7f4af51c8325689758f86541b865528ea7e Nov 04 05:45:55.691158 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.691126 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:45:55.691235 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.691189 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:45:55.691235 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:55.691217 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:45:56.224496 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:56.224459 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" event={"ID":"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d","Type":"ContainerStarted","Data":"14be97e38171ff294c427a7eee274515208a7dc34c7bc33654dcabeea28fbc2a"} Nov 04 05:45:56.224714 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:56.224510 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" event={"ID":"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d","Type":"ContainerStarted","Data":"a6c371ff99f9730d1ddf6620524bd7f4af51c8325689758f86541b865528ea7e"} Nov 04 05:45:56.239871 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:56.239848 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:45:56.241525 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:56.241499 4859 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Nov 04 05:45:56.241621 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:56.241559 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" podUID="2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Nov 04 05:45:56.354554 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:56.354502 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" podStartSLOduration=2.354485833 podStartE2EDuration="2.354485833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:56.353428096 +0000 UTC m=+1126.355063995" watchObservedRunningTime="2026-11-04 05:45:56.354485833 +0000 UTC m=+1126.356121558" Nov 04 05:45:57.016233 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.016198 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm"] Nov 04 05:45:57.016819 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.016802 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="6fda8c73-74bb-49ce-b03e-e5517015ee6f" containerName="istio-proxy" Nov 04 05:45:57.020095 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.020074 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.143016 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.142971 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm"] Nov 04 05:45:57.153102 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.153071 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.153267 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.153115 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.153267 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.153212 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.153342 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.153290 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.153342 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.153321 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.153404 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.153359 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8018ad9e-115b-47b3-9475-e6b06e8de89f-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.153478 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.153459 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.153512 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.153495 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.153613 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.153592 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqdkp\" (UniqueName: \"kubernetes.io/projected/8018ad9e-115b-47b3-9475-e6b06e8de89f-kube-api-access-cqdkp\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.239280 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.239251 4859 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Nov 04 05:45:57.239436 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.239307 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" podUID="2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Nov 04 05:45:57.254746 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.254721 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqdkp\" (UniqueName: \"kubernetes.io/projected/8018ad9e-115b-47b3-9475-e6b06e8de89f-kube-api-access-cqdkp\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.254853 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.254811 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.254853 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.254836 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.254933 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.254870 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.254984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.254928 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.255039 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.254990 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.255096 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.255051 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8018ad9e-115b-47b3-9475-e6b06e8de89f-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.255193 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.255145 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.255269 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.255211 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.255328 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.255268 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.255385 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.255332 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.255467 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.255445 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.255541 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.255486 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.255890 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.255868 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8018ad9e-115b-47b3-9475-e6b06e8de89f-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.257379 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.257357 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.257464 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.257425 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.320262 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.320195 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.320435 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.320413 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqdkp\" (UniqueName: \"kubernetes.io/projected/8018ad9e-115b-47b3-9475-e6b06e8de89f-kube-api-access-cqdkp\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.334091 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.334070 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:57.808204 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.808146 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm"] Nov 04 05:45:57.810642 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:45:57.810610 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8018ad9e_115b_47b3_9475_e6b06e8de89f.slice/crio-64028dd1252cb6c434e3d14dfa079235e8f9332ac2db4d06bb3cce8134077dac WatchSource:0}: Error finding container 64028dd1252cb6c434e3d14dfa079235e8f9332ac2db4d06bb3cce8134077dac: Status 404 returned error can't find the container with id 64028dd1252cb6c434e3d14dfa079235e8f9332ac2db4d06bb3cce8134077dac Nov 04 05:45:57.812999 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.812967 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:45:57.813091 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.813031 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:45:57.813091 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:57.813061 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:45:58.236241 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:58.236206 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" event={"ID":"8018ad9e-115b-47b3-9475-e6b06e8de89f","Type":"ContainerStarted","Data":"fd3328be4b32ed7e7db82b3f40fd95bff8c0cee82cd2e0c7f905a2d275d0b725"} Nov 04 05:45:58.236637 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:58.236277 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" event={"ID":"8018ad9e-115b-47b3-9475-e6b06e8de89f","Type":"ContainerStarted","Data":"64028dd1252cb6c434e3d14dfa079235e8f9332ac2db4d06bb3cce8134077dac"} Nov 04 05:45:58.239918 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:58.239889 4859 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Nov 04 05:45:58.240024 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:58.239946 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" podUID="2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Nov 04 05:45:58.335719 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:58.335133 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:45:58.348076 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:58.347939 4859 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Nov 04 05:45:58.348076 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:58.348009 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" podUID="8018ad9e-115b-47b3-9475-e6b06e8de89f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Nov 04 05:45:58.369508 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:58.369418 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" podStartSLOduration=2.369401436 podStartE2EDuration="2.369401436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:58.36872042 +0000 UTC m=+1128.370356148" watchObservedRunningTime="2026-11-04 05:45:58.369401436 +0000 UTC m=+1128.371037163" Nov 04 05:45:58.383850 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:45:58.372212 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:45:59.239444 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:59.239405 4859 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Nov 04 05:45:59.239939 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:59.239470 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" podUID="2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Nov 04 05:45:59.334649 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:59.334616 4859 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Nov 04 05:45:59.334838 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:45:59.334691 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" podUID="8018ad9e-115b-47b3-9475-e6b06e8de89f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Nov 04 05:46:00.246592 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:00.246561 4859 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Nov 04 05:46:00.247694 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:00.247637 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" podUID="2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Nov 04 05:46:00.334455 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:00.334423 4859 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Nov 04 05:46:00.334739 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:00.334700 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" podUID="8018ad9e-115b-47b3-9475-e6b06e8de89f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Nov 04 05:46:01.239916 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:01.239882 4859 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Nov 04 05:46:01.240097 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:01.239936 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" podUID="2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Nov 04 05:46:01.335414 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:01.335378 4859 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Nov 04 05:46:01.335816 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:01.335428 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" podUID="8018ad9e-115b-47b3-9475-e6b06e8de89f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Nov 04 05:46:02.239166 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:02.239130 4859 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Nov 04 05:46:02.239355 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:02.239193 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" podUID="2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Nov 04 05:46:02.335280 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:02.335249 4859 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Nov 04 05:46:02.335482 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:02.335301 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" podUID="8018ad9e-115b-47b3-9475-e6b06e8de89f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Nov 04 05:46:03.335295 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:03.335253 4859 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Nov 04 05:46:03.335520 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:03.335318 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" podUID="8018ad9e-115b-47b3-9475-e6b06e8de89f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Nov 04 05:46:03.344192 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:03.344163 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:46:03.344992 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:03.344957 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:46:03.346521 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:03.346499 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:46:04.239482 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.239426 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" Nov 04 05:46:04.305428 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.305389 4859 generic.go:356] "Generic (PLEG): container finished" podID="74964f80-4702-460f-95bd-864bd6b30d93" containerID="0408db1f14fccd416dd3e51d7646370b1dd402c89da0abec345839d03078b8de" exitCode=0 Nov 04 05:46:04.305582 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.305463 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" event={"ID":"74964f80-4702-460f-95bd-864bd6b30d93","Type":"ContainerDied","Data":"0408db1f14fccd416dd3e51d7646370b1dd402c89da0abec345839d03078b8de"} Nov 04 05:46:04.307712 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.306943 4859 generic.go:356] "Generic (PLEG): container finished" podID="3567dc66-3594-4163-a4d1-e2a4e9d5b408" containerID="3db1fa2a750118eceabe9cc94366cd6d10cc21732de992558a3b07c2f4b257b6" exitCode=0 Nov 04 05:46:04.308117 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.307868 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" Nov 04 05:46:04.308117 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.307975 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" event={"ID":"3567dc66-3594-4163-a4d1-e2a4e9d5b408","Type":"ContainerDied","Data":"3db1fa2a750118eceabe9cc94366cd6d10cc21732de992558a3b07c2f4b257b6"} Nov 04 05:46:04.308117 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.308004 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746" event={"ID":"3567dc66-3594-4163-a4d1-e2a4e9d5b408","Type":"ContainerDied","Data":"72ab87ebc13aa6d1c683cf63187cd7a80e5dfddb257149b59e519a0ef42db4cd"} Nov 04 05:46:04.308117 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.308025 4859 scope.go:122] "RemoveContainer" containerID="3db1fa2a750118eceabe9cc94366cd6d10cc21732de992558a3b07c2f4b257b6" Nov 04 05:46:04.320164 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.320144 4859 scope.go:122] "RemoveContainer" containerID="3db1fa2a750118eceabe9cc94366cd6d10cc21732de992558a3b07c2f4b257b6" Nov 04 05:46:04.320501 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:46:04.320469 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3db1fa2a750118eceabe9cc94366cd6d10cc21732de992558a3b07c2f4b257b6\": container with ID starting with 3db1fa2a750118eceabe9cc94366cd6d10cc21732de992558a3b07c2f4b257b6 not found: ID does not exist" containerID="3db1fa2a750118eceabe9cc94366cd6d10cc21732de992558a3b07c2f4b257b6" Nov 04 05:46:04.320628 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.320511 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3db1fa2a750118eceabe9cc94366cd6d10cc21732de992558a3b07c2f4b257b6"} err="failed to get container status \"3db1fa2a750118eceabe9cc94366cd6d10cc21732de992558a3b07c2f4b257b6\": rpc error: code = NotFound desc = could not find container \"3db1fa2a750118eceabe9cc94366cd6d10cc21732de992558a3b07c2f4b257b6\": container with ID starting with 3db1fa2a750118eceabe9cc94366cd6d10cc21732de992558a3b07c2f4b257b6 not found: ID does not exist" Nov 04 05:46:04.334969 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.334945 4859 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Nov 04 05:46:04.335088 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.334995 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" podUID="8018ad9e-115b-47b3-9475-e6b06e8de89f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Nov 04 05:46:04.374247 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.374219 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3567dc66-3594-4163-a4d1-e2a4e9d5b408-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3567dc66-3594-4163-a4d1-e2a4e9d5b408-plugins-config-volume\") pod \"3567dc66-3594-4163-a4d1-e2a4e9d5b408\" (UID: \"3567dc66-3594-4163-a4d1-e2a4e9d5b408\") " Nov 04 05:46:04.374599 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.374329 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3567dc66-3594-4163-a4d1-e2a4e9d5b408-kube-api-access-b8fnp\" (UniqueName: \"kubernetes.io/projected/3567dc66-3594-4163-a4d1-e2a4e9d5b408-kube-api-access-b8fnp\") pod \"3567dc66-3594-4163-a4d1-e2a4e9d5b408\" (UID: \"3567dc66-3594-4163-a4d1-e2a4e9d5b408\") " Nov 04 05:46:04.374643 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.374616 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3567dc66-3594-4163-a4d1-e2a4e9d5b408-plugins-config-volume" pod "3567dc66-3594-4163-a4d1-e2a4e9d5b408" (UID: "3567dc66-3594-4163-a4d1-e2a4e9d5b408"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:04.374759 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.374745 4859 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3567dc66-3594-4163-a4d1-e2a4e9d5b408-plugins-config-volume\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:04.375439 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.375419 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" Nov 04 05:46:04.376366 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.376346 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3567dc66-3594-4163-a4d1-e2a4e9d5b408-kube-api-access-b8fnp" pod "3567dc66-3594-4163-a4d1-e2a4e9d5b408" (UID: "3567dc66-3594-4163-a4d1-e2a4e9d5b408"). InnerVolumeSpecName "kube-api-access-b8fnp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:04.475747 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.475717 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74964f80-4702-460f-95bd-864bd6b30d93-kube-api-access-7swfz\" (UniqueName: \"kubernetes.io/projected/74964f80-4702-460f-95bd-864bd6b30d93-kube-api-access-7swfz\") pod \"74964f80-4702-460f-95bd-864bd6b30d93\" (UID: \"74964f80-4702-460f-95bd-864bd6b30d93\") " Nov 04 05:46:04.475937 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.475856 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74964f80-4702-460f-95bd-864bd6b30d93-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74964f80-4702-460f-95bd-864bd6b30d93-plugins-config-volume\") pod \"74964f80-4702-460f-95bd-864bd6b30d93\" (UID: \"74964f80-4702-460f-95bd-864bd6b30d93\") " Nov 04 05:46:04.476455 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.476219 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8fnp\" (UniqueName: \"kubernetes.io/projected/3567dc66-3594-4163-a4d1-e2a4e9d5b408-kube-api-access-b8fnp\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:04.476455 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.476242 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74964f80-4702-460f-95bd-864bd6b30d93-plugins-config-volume" pod "74964f80-4702-460f-95bd-864bd6b30d93" (UID: "74964f80-4702-460f-95bd-864bd6b30d93"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:04.477934 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.477912 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74964f80-4702-460f-95bd-864bd6b30d93-kube-api-access-7swfz" pod "74964f80-4702-460f-95bd-864bd6b30d93" (UID: "74964f80-4702-460f-95bd-864bd6b30d93"). InnerVolumeSpecName "kube-api-access-7swfz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:04.580818 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.580725 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7swfz\" (UniqueName: \"kubernetes.io/projected/74964f80-4702-460f-95bd-864bd6b30d93-kube-api-access-7swfz\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:04.580818 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.580764 4859 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74964f80-4702-460f-95bd-864bd6b30d93-plugins-config-volume\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:04.660573 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.660539 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx"] Nov 04 05:46:04.660894 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.660863 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" podUID="4645be22-7060-4e20-9fde-6f7595ef45da" containerName="maas-api" containerID="cri-o://9e8a31add99f872d7fe607f1918fad6a8c9432c517f7797aefcf6358a416f105" gracePeriod=30 Nov 04 05:46:04.737869 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.737832 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746"] Nov 04 05:46:04.742485 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:04.742456 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-a84d48-55855658b-sl746"] Nov 04 05:46:05.315558 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:05.315459 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" event={"ID":"74964f80-4702-460f-95bd-864bd6b30d93","Type":"ContainerDied","Data":"000395a6180f4a5c5f6b33fc99079df919160d4cbdb5362ff2a5fcc0b865d018"} Nov 04 05:46:05.315558 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:05.315503 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l" Nov 04 05:46:05.315558 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:05.315539 4859 scope.go:122] "RemoveContainer" containerID="0408db1f14fccd416dd3e51d7646370b1dd402c89da0abec345839d03078b8de" Nov 04 05:46:05.452656 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:05.452624 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:46:05.453480 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:05.453458 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:46:05.454507 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:05.454482 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:46:05.495725 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:05.495695 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l"] Nov 04 05:46:05.498966 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:05.498943 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-a84d48-5bdfff8879-gp67l"] Nov 04 05:46:05.980961 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:05.980929 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:46:05.998354 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:05.998322 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-maas-api-tls\") pod \"4645be22-7060-4e20-9fde-6f7595ef45da\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " Nov 04 05:46:05.998525 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:05.998396 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4645be22-7060-4e20-9fde-6f7595ef45da-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4645be22-7060-4e20-9fde-6f7595ef45da-openshift-service-ca\") pod \"4645be22-7060-4e20-9fde-6f7595ef45da\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " Nov 04 05:46:05.998525 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:05.998515 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-metrics-certs\") pod \"4645be22-7060-4e20-9fde-6f7595ef45da\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " Nov 04 05:46:05.998655 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:05.998556 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4645be22-7060-4e20-9fde-6f7595ef45da-kube-api-access-5ftrn\" (UniqueName: \"kubernetes.io/projected/4645be22-7060-4e20-9fde-6f7595ef45da-kube-api-access-5ftrn\") pod \"4645be22-7060-4e20-9fde-6f7595ef45da\" (UID: \"4645be22-7060-4e20-9fde-6f7595ef45da\") " Nov 04 05:46:05.999254 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:05.999156 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4645be22-7060-4e20-9fde-6f7595ef45da-openshift-service-ca" pod "4645be22-7060-4e20-9fde-6f7595ef45da" (UID: "4645be22-7060-4e20-9fde-6f7595ef45da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:06.001390 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.001351 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-maas-api-tls" pod "4645be22-7060-4e20-9fde-6f7595ef45da" (UID: "4645be22-7060-4e20-9fde-6f7595ef45da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:46:06.001783 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.001758 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4645be22-7060-4e20-9fde-6f7595ef45da-kube-api-access-5ftrn" pod "4645be22-7060-4e20-9fde-6f7595ef45da" (UID: "4645be22-7060-4e20-9fde-6f7595ef45da"). InnerVolumeSpecName "kube-api-access-5ftrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:06.001886 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.001658 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-metrics-certs" pod "4645be22-7060-4e20-9fde-6f7595ef45da" (UID: "4645be22-7060-4e20-9fde-6f7595ef45da"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:46:06.100158 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.100116 4859 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4645be22-7060-4e20-9fde-6f7595ef45da-openshift-service-ca\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:06.100158 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.100155 4859 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-metrics-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:06.100433 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.100172 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ftrn\" (UniqueName: \"kubernetes.io/projected/4645be22-7060-4e20-9fde-6f7595ef45da-kube-api-access-5ftrn\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:06.100433 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.100187 4859 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4645be22-7060-4e20-9fde-6f7595ef45da-maas-api-tls\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:06.122074 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.122031 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3567dc66-3594-4163-a4d1-e2a4e9d5b408" path="/var/lib/kubelet/pods/3567dc66-3594-4163-a4d1-e2a4e9d5b408/volumes" Nov 04 05:46:06.122635 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.122614 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74964f80-4702-460f-95bd-864bd6b30d93" path="/var/lib/kubelet/pods/74964f80-4702-460f-95bd-864bd6b30d93/volumes" Nov 04 05:46:06.325811 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.325729 4859 generic.go:356] "Generic (PLEG): container finished" podID="4645be22-7060-4e20-9fde-6f7595ef45da" containerID="9e8a31add99f872d7fe607f1918fad6a8c9432c517f7797aefcf6358a416f105" exitCode=0 Nov 04 05:46:06.325811 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.325766 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" event={"ID":"4645be22-7060-4e20-9fde-6f7595ef45da","Type":"ContainerDied","Data":"9e8a31add99f872d7fe607f1918fad6a8c9432c517f7797aefcf6358a416f105"} Nov 04 05:46:06.325811 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.325802 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" Nov 04 05:46:06.326090 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.325837 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx" event={"ID":"4645be22-7060-4e20-9fde-6f7595ef45da","Type":"ContainerDied","Data":"2bc0846e398da387ab80499c9f1c39e06b724db5557dfa553a7745ca1941a975"} Nov 04 05:46:06.326090 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.325870 4859 scope.go:122] "RemoveContainer" containerID="9e8a31add99f872d7fe607f1918fad6a8c9432c517f7797aefcf6358a416f105" Nov 04 05:46:06.337621 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.337597 4859 scope.go:122] "RemoveContainer" containerID="9e8a31add99f872d7fe607f1918fad6a8c9432c517f7797aefcf6358a416f105" Nov 04 05:46:06.337963 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:46:06.337931 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e8a31add99f872d7fe607f1918fad6a8c9432c517f7797aefcf6358a416f105\": container with ID starting with 9e8a31add99f872d7fe607f1918fad6a8c9432c517f7797aefcf6358a416f105 not found: ID does not exist" containerID="9e8a31add99f872d7fe607f1918fad6a8c9432c517f7797aefcf6358a416f105" Nov 04 05:46:06.338111 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.337969 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e8a31add99f872d7fe607f1918fad6a8c9432c517f7797aefcf6358a416f105"} err="failed to get container status \"9e8a31add99f872d7fe607f1918fad6a8c9432c517f7797aefcf6358a416f105\": rpc error: code = NotFound desc = could not find container \"9e8a31add99f872d7fe607f1918fad6a8c9432c517f7797aefcf6358a416f105\": container with ID starting with 9e8a31add99f872d7fe607f1918fad6a8c9432c517f7797aefcf6358a416f105 not found: ID does not exist" Nov 04 05:46:06.506924 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.506889 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx"] Nov 04 05:46:06.559473 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:06.559445 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-a84d48-56ff9b7d89-flxhx"] Nov 04 05:46:08.122694 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:08.122414 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4645be22-7060-4e20-9fde-6f7595ef45da" path="/var/lib/kubelet/pods/4645be22-7060-4e20-9fde-6f7595ef45da/volumes" Nov 04 05:46:10.275282 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:10.275253 4859 scope.go:122] "RemoveContainer" containerID="1cc40cae53d8feb15552a5ed5847eda217a1c90c06c65d752cd5a135b111a7d5" Nov 04 05:46:17.170332 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:17.170296 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv"] Nov 04 05:46:17.170789 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:17.170501 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" podUID="0e75fb43-a2fa-48d1-bc71-8f4f3514b817" containerName="istio-proxy" containerID="cri-o://93ea2b28bd9cc20f3d9efd61b71b43c1e95c84958a63c1299fc995657b81060e" gracePeriod=30 Nov 04 05:46:17.189331 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:17.189301 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9"] Nov 04 05:46:17.189529 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:17.189495 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" podUID="2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" containerName="istio-proxy" containerID="cri-o://14be97e38171ff294c427a7eee274515208a7dc34c7bc33654dcabeea28fbc2a" gracePeriod=30 Nov 04 05:46:18.347372 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:18.347340 4859 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:46:18.347760 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:18.347404 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" podUID="2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:46:22.422997 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.422967 4859 generic.go:356] "Generic (PLEG): container finished" podID="0e75fb43-a2fa-48d1-bc71-8f4f3514b817" containerID="93ea2b28bd9cc20f3d9efd61b71b43c1e95c84958a63c1299fc995657b81060e" exitCode=0 Nov 04 05:46:22.423420 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.423050 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" event={"ID":"0e75fb43-a2fa-48d1-bc71-8f4f3514b817","Type":"ContainerDied","Data":"93ea2b28bd9cc20f3d9efd61b71b43c1e95c84958a63c1299fc995657b81060e"} Nov 04 05:46:22.425271 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.425242 4859 generic.go:356] "Generic (PLEG): container finished" podID="2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" containerID="14be97e38171ff294c427a7eee274515208a7dc34c7bc33654dcabeea28fbc2a" exitCode=0 Nov 04 05:46:22.425386 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.425287 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" event={"ID":"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d","Type":"ContainerDied","Data":"14be97e38171ff294c427a7eee274515208a7dc34c7bc33654dcabeea28fbc2a"} Nov 04 05:46:22.449650 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.449601 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:46:22.475952 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.475932 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:46:22.590770 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.590739 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-kube-api-access-gnlcv\" (UniqueName: \"kubernetes.io/projected/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-kube-api-access-gnlcv\") pod \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " Nov 04 05:46:22.590930 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.590809 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-token\" (UniqueName: \"kubernetes.io/projected/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-token\") pod \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " Nov 04 05:46:22.590930 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.590854 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-envoy\") pod \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " Nov 04 05:46:22.590930 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.590887 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istiod-ca-cert\") pod \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " Nov 04 05:46:22.590930 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.590914 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-credential-socket\") pod \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " Nov 04 05:46:22.591143 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.590940 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-data\") pod \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " Nov 04 05:46:22.591143 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.590994 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-podinfo\") pod \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " Nov 04 05:46:22.591143 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.591019 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-token\" (UniqueName: \"kubernetes.io/projected/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-token\") pod \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " Nov 04 05:46:22.591143 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.591062 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-workload-socket\") pod \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " Nov 04 05:46:22.591143 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.591124 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istiod-ca-cert\") pod \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " Nov 04 05:46:22.591426 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.591165 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-workload-certs\") pod \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " Nov 04 05:46:22.591426 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.591218 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-credential-socket\") pod \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " Nov 04 05:46:22.591426 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.591260 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-envoy\") pod \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " Nov 04 05:46:22.591426 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.591293 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-podinfo\") pod \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " Nov 04 05:46:22.591426 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.591358 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-data\") pod \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " Nov 04 05:46:22.591426 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.591351 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istiod-ca-cert" pod "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" (UID: "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:22.591765 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.591466 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-kube-api-access-pp6xr\" (UniqueName: \"kubernetes.io/projected/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-kube-api-access-pp6xr\") pod \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\" (UID: \"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d\") " Nov 04 05:46:22.591765 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.591615 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-data" pod "0e75fb43-a2fa-48d1-bc71-8f4f3514b817" (UID: "0e75fb43-a2fa-48d1-bc71-8f4f3514b817"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.591765 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.591656 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-credential-socket" pod "0e75fb43-a2fa-48d1-bc71-8f4f3514b817" (UID: "0e75fb43-a2fa-48d1-bc71-8f4f3514b817"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.591928 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.591884 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-workload-certs" pod "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" (UID: "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.592893 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.592118 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-workload-socket" pod "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" (UID: "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.592893 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.592418 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istiod-ca-cert" pod "0e75fb43-a2fa-48d1-bc71-8f4f3514b817" (UID: "0e75fb43-a2fa-48d1-bc71-8f4f3514b817"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:22.592893 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.592677 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-data" pod "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" (UID: "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.592893 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.592858 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-credential-socket" pod "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" (UID: "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.593656 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.593379 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-workload-socket" pod "0e75fb43-a2fa-48d1-bc71-8f4f3514b817" (UID: "0e75fb43-a2fa-48d1-bc71-8f4f3514b817"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.593656 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.593430 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-kube-api-access-gnlcv" pod "0e75fb43-a2fa-48d1-bc71-8f4f3514b817" (UID: "0e75fb43-a2fa-48d1-bc71-8f4f3514b817"). InnerVolumeSpecName "kube-api-access-gnlcv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:22.593813 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.591583 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-workload-socket\") pod \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " Nov 04 05:46:22.593946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.593926 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-workload-certs\") pod \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\" (UID: \"0e75fb43-a2fa-48d1-bc71-8f4f3514b817\") " Nov 04 05:46:22.594518 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594225 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-workload-certs" pod "0e75fb43-a2fa-48d1-bc71-8f4f3514b817" (UID: "0e75fb43-a2fa-48d1-bc71-8f4f3514b817"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.594518 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594380 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-envoy" pod "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" (UID: "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.594518 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594446 4859 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-workload-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.594518 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594464 4859 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-workload-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.594518 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594480 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnlcv\" (UniqueName: \"kubernetes.io/projected/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-kube-api-access-gnlcv\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.594518 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594494 4859 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istiod-ca-cert\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.594518 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594496 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-podinfo" pod "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" (UID: "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:46:22.594853 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594507 4859 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-credential-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.594853 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594548 4859 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-data\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.594853 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594565 4859 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-workload-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.594853 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594574 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-token" pod "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" (UID: "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:22.594853 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594580 4859 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istiod-ca-cert\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.594853 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594612 4859 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-workload-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.594853 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594629 4859 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-credential-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.594853 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594644 4859 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-data\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.594853 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594752 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-token" pod "0e75fb43-a2fa-48d1-bc71-8f4f3514b817" (UID: "0e75fb43-a2fa-48d1-bc71-8f4f3514b817"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:22.595136 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.594897 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-envoy" pod "0e75fb43-a2fa-48d1-bc71-8f4f3514b817" (UID: "0e75fb43-a2fa-48d1-bc71-8f4f3514b817"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.595420 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.595399 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-podinfo" pod "0e75fb43-a2fa-48d1-bc71-8f4f3514b817" (UID: "0e75fb43-a2fa-48d1-bc71-8f4f3514b817"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:46:22.595528 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.595479 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-kube-api-access-pp6xr" pod "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" (UID: "2912c2d6-a4ad-4253-b9fa-f51c26afdd8d"). InnerVolumeSpecName "kube-api-access-pp6xr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:22.696158 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.696127 4859 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-podinfo\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.696273 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.696162 4859 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.696273 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.696179 4859 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e75fb43-a2fa-48d1-bc71-8f4f3514b817-istio-envoy\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.696273 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.696194 4859 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-podinfo\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.696273 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.696208 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp6xr\" (UniqueName: \"kubernetes.io/projected/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-kube-api-access-pp6xr\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.696273 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.696223 4859 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.696273 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:22.696236 4859 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d-istio-envoy\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:23.432440 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:23.432404 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" event={"ID":"2912c2d6-a4ad-4253-b9fa-f51c26afdd8d","Type":"ContainerDied","Data":"a6c371ff99f9730d1ddf6620524bd7f4af51c8325689758f86541b865528ea7e"} Nov 04 05:46:23.432440 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:23.432424 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9" Nov 04 05:46:23.432986 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:23.432502 4859 scope.go:122] "RemoveContainer" containerID="14be97e38171ff294c427a7eee274515208a7dc34c7bc33654dcabeea28fbc2a" Nov 04 05:46:23.433941 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:23.433925 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" Nov 04 05:46:23.434023 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:23.433924 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv" event={"ID":"0e75fb43-a2fa-48d1-bc71-8f4f3514b817","Type":"ContainerDied","Data":"4b6f65268ef355d36fe55e81e6d3a0a028ea9792b6d57fce988b272c36c11f6d"} Nov 04 05:46:23.442212 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:23.442193 4859 scope.go:122] "RemoveContainer" containerID="93ea2b28bd9cc20f3d9efd61b71b43c1e95c84958a63c1299fc995657b81060e" Nov 04 05:46:23.616423 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:23.616390 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv"] Nov 04 05:46:23.670487 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:23.670461 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-mh4zv"] Nov 04 05:46:23.738785 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:23.738705 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9"] Nov 04 05:46:23.742833 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:23.742808 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-x65f9"] Nov 04 05:46:24.115446 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:24.115372 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e75fb43-a2fa-48d1-bc71-8f4f3514b817" path="/var/lib/kubelet/pods/0e75fb43-a2fa-48d1-bc71-8f4f3514b817/volumes" Nov 04 05:46:24.115832 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:24.115819 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" path="/var/lib/kubelet/pods/2912c2d6-a4ad-4253-b9fa-f51c26afdd8d/volumes" Nov 04 05:46:26.896690 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.896629 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd"] Nov 04 05:46:26.897876 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.897848 4859 cpu_manager.go:414] "containerMap: removing container" podUID="0e75fb43-a2fa-48d1-bc71-8f4f3514b817" containerName="istio-proxy" Nov 04 05:46:26.897876 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.897876 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e75fb43-a2fa-48d1-bc71-8f4f3514b817" containerName="istio-proxy" Nov 04 05:46:26.898079 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.897933 4859 cpu_manager.go:414] "containerMap: removing container" podUID="4645be22-7060-4e20-9fde-6f7595ef45da" containerName="maas-api" Nov 04 05:46:26.898079 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.897942 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4645be22-7060-4e20-9fde-6f7595ef45da" containerName="maas-api" Nov 04 05:46:26.898079 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.897987 4859 cpu_manager.go:414] "containerMap: removing container" podUID="3567dc66-3594-4163-a4d1-e2a4e9d5b408" containerName="payload-processing" Nov 04 05:46:26.898079 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.897996 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3567dc66-3594-4163-a4d1-e2a4e9d5b408" containerName="payload-processing" Nov 04 05:46:26.898079 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.898026 4859 cpu_manager.go:414] "containerMap: removing container" podUID="2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" containerName="istio-proxy" Nov 04 05:46:26.898079 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.898038 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" containerName="istio-proxy" Nov 04 05:46:26.898079 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.898050 4859 cpu_manager.go:414] "containerMap: removing container" podUID="74964f80-4702-460f-95bd-864bd6b30d93" containerName="payload-pre-processing" Nov 04 05:46:26.898079 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.898058 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74964f80-4702-460f-95bd-864bd6b30d93" containerName="payload-pre-processing" Nov 04 05:46:26.898447 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.898273 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="2912c2d6-a4ad-4253-b9fa-f51c26afdd8d" containerName="istio-proxy" Nov 04 05:46:26.898447 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.898298 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="74964f80-4702-460f-95bd-864bd6b30d93" containerName="payload-pre-processing" Nov 04 05:46:26.898447 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.898311 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e75fb43-a2fa-48d1-bc71-8f4f3514b817" containerName="istio-proxy" Nov 04 05:46:26.898447 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.898325 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="4645be22-7060-4e20-9fde-6f7595ef45da" containerName="maas-api" Nov 04 05:46:26.898447 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.898348 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="3567dc66-3594-4163-a4d1-e2a4e9d5b408" containerName="payload-processing" Nov 04 05:46:26.902836 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.902707 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:26.950463 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.950433 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj"] Nov 04 05:46:26.954148 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.954131 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:26.964879 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:26.964859 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-901a00-openshift-default-dockercfg-txfc4\"" Nov 04 05:46:27.014948 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.014918 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd"] Nov 04 05:46:27.022812 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.022788 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj"] Nov 04 05:46:27.040563 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.040536 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.040720 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.040580 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.040720 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.040616 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.040823 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.040795 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.040865 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.040851 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.040930 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.040908 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.040966 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.040956 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.041025 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.041009 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3635c9-2a57-4ebb-a371-877a21946f46-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.041058 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.041034 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.041089 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.041058 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.041157 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.041141 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.041190 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.041166 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjm75\" (UniqueName: \"kubernetes.io/projected/2d4d34a1-0909-4690-8b21-e90d76afdfde-kube-api-access-qjm75\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.041244 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.041219 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.041341 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.041321 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.041419 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.041403 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhxfv\" (UniqueName: \"kubernetes.io/projected/6b3635c9-2a57-4ebb-a371-877a21946f46-kube-api-access-mhxfv\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.041450 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.041441 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.041483 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.041473 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d4d34a1-0909-4690-8b21-e90d76afdfde-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.041538 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.041522 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.142419 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142382 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.142582 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142427 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.142582 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142451 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjm75\" (UniqueName: \"kubernetes.io/projected/2d4d34a1-0909-4690-8b21-e90d76afdfde-kube-api-access-qjm75\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.142582 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142477 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.142582 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142506 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.142582 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142555 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhxfv\" (UniqueName: \"kubernetes.io/projected/6b3635c9-2a57-4ebb-a371-877a21946f46-kube-api-access-mhxfv\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.142905 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142586 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.142905 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142631 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d4d34a1-0909-4690-8b21-e90d76afdfde-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.142905 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142689 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.142905 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142722 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.142905 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142755 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.142905 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142791 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.142905 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142827 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.142905 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142836 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.142905 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142867 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.143346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142919 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.143346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142973 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.143346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.142979 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.143346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.143006 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3635c9-2a57-4ebb-a371-877a21946f46-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.143346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.143034 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.143346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.143065 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.143346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.143102 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.143346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.143170 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.143773 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.143363 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.143773 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.143421 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.143773 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.143726 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.144068 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.144047 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3635c9-2a57-4ebb-a371-877a21946f46-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.144154 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.144118 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d4d34a1-0909-4690-8b21-e90d76afdfde-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.145291 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.145270 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.145380 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.145346 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.145499 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.145483 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.145604 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.145585 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.208083 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.208052 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.208083 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.208076 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhxfv\" (UniqueName: \"kubernetes.io/projected/6b3635c9-2a57-4ebb-a371-877a21946f46-kube-api-access-mhxfv\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.208243 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.208148 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.208243 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.208209 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjm75\" (UniqueName: \"kubernetes.io/projected/2d4d34a1-0909-4690-8b21-e90d76afdfde-kube-api-access-qjm75\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.216625 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.216607 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:27.267382 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.267353 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:27.665970 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.665939 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd"] Nov 04 05:46:27.669418 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:46:27.669392 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d4d34a1_0909_4690_8b21_e90d76afdfde.slice/crio-3bcbc515e8ff6609d8f32d8944fe52f0f68737353e932938295d3c1d371c255c WatchSource:0}: Error finding container 3bcbc515e8ff6609d8f32d8944fe52f0f68737353e932938295d3c1d371c255c: Status 404 returned error can't find the container with id 3bcbc515e8ff6609d8f32d8944fe52f0f68737353e932938295d3c1d371c255c Nov 04 05:46:27.671810 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.671772 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:46:27.671897 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.671859 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:46:27.671958 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.671902 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:46:27.726265 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.726242 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj"] Nov 04 05:46:27.729858 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:46:27.729833 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b3635c9_2a57_4ebb_a371_877a21946f46.slice/crio-b6435b6fadcff9c5557178133f29c4b40b8feac586489f5e5b005a1a9ace2731 WatchSource:0}: Error finding container b6435b6fadcff9c5557178133f29c4b40b8feac586489f5e5b005a1a9ace2731: Status 404 returned error can't find the container with id b6435b6fadcff9c5557178133f29c4b40b8feac586489f5e5b005a1a9ace2731 Nov 04 05:46:27.732005 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.731967 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:46:27.732084 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.732033 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:46:27.732084 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:27.732065 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:46:28.364847 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:46:28.364809 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:46:28.462444 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:28.462401 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" event={"ID":"6b3635c9-2a57-4ebb-a371-877a21946f46","Type":"ContainerStarted","Data":"05d563eba0a844b227d0c111f4e1368256207a7b7de9934fe437f5be193d4889"} Nov 04 05:46:28.462643 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:28.462464 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" event={"ID":"6b3635c9-2a57-4ebb-a371-877a21946f46","Type":"ContainerStarted","Data":"b6435b6fadcff9c5557178133f29c4b40b8feac586489f5e5b005a1a9ace2731"} Nov 04 05:46:28.464098 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:28.464075 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" event={"ID":"2d4d34a1-0909-4690-8b21-e90d76afdfde","Type":"ContainerStarted","Data":"ef7b3c953a7643bc1b84e8cca637100d151954ed127bee8c0f49a885d8f2d08b"} Nov 04 05:46:28.464207 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:28.464113 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" event={"ID":"2d4d34a1-0909-4690-8b21-e90d76afdfde","Type":"ContainerStarted","Data":"3bcbc515e8ff6609d8f32d8944fe52f0f68737353e932938295d3c1d371c255c"} Nov 04 05:46:28.614688 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:28.600522 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" podStartSLOduration=2.600501827 podStartE2EDuration="2.600501827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:46:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:46:28.598933519 +0000 UTC m=+1158.600569245" watchObservedRunningTime="2026-11-04 05:46:28.600501827 +0000 UTC m=+1158.602137553" Nov 04 05:46:28.729689 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:28.728502 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" podStartSLOduration=2.72848496 podStartE2EDuration="2.72848496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:46:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:46:28.727179013 +0000 UTC m=+1158.728814751" watchObservedRunningTime="2026-11-04 05:46:28.72848496 +0000 UTC m=+1158.730120688" Nov 04 05:46:29.219998 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:29.218010 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:29.220696 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:29.220322 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Nov 04 05:46:29.220696 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:29.220395 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" podUID="2d4d34a1-0909-4690-8b21-e90d76afdfde" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Nov 04 05:46:29.268439 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:29.268415 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:29.269679 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:29.269645 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Nov 04 05:46:29.269845 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:29.269822 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" podUID="6b3635c9-2a57-4ebb-a371-877a21946f46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Nov 04 05:46:30.223689 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:30.223119 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Nov 04 05:46:30.223689 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:30.223187 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" podUID="2d4d34a1-0909-4690-8b21-e90d76afdfde" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Nov 04 05:46:30.268186 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:30.268156 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Nov 04 05:46:30.268346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:30.268217 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" podUID="6b3635c9-2a57-4ebb-a371-877a21946f46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Nov 04 05:46:31.222102 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:31.222061 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Nov 04 05:46:31.222512 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:31.222480 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" podUID="2d4d34a1-0909-4690-8b21-e90d76afdfde" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Nov 04 05:46:31.268212 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:31.268178 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Nov 04 05:46:31.268568 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:31.268220 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" podUID="6b3635c9-2a57-4ebb-a371-877a21946f46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Nov 04 05:46:32.217126 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:32.217088 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Nov 04 05:46:32.217410 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:32.217379 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" podUID="2d4d34a1-0909-4690-8b21-e90d76afdfde" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Nov 04 05:46:32.267845 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:32.267813 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Nov 04 05:46:32.267995 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:32.267866 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" podUID="6b3635c9-2a57-4ebb-a371-877a21946f46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Nov 04 05:46:32.529037 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:32.528948 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8"] Nov 04 05:46:33.218185 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:33.218148 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Nov 04 05:46:33.218502 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:33.218452 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" podUID="2d4d34a1-0909-4690-8b21-e90d76afdfde" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Nov 04 05:46:33.267675 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:33.267641 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Nov 04 05:46:33.267823 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:33.267748 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" podUID="6b3635c9-2a57-4ebb-a371-877a21946f46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Nov 04 05:46:34.217788 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:34.217754 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Nov 04 05:46:34.218177 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:34.217820 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" podUID="2d4d34a1-0909-4690-8b21-e90d76afdfde" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Nov 04 05:46:34.267744 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:34.267705 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Nov 04 05:46:34.267936 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:34.267769 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" podUID="6b3635c9-2a57-4ebb-a371-877a21946f46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Nov 04 05:46:35.220886 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:35.220857 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:35.221270 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:35.221233 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:35.222029 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:35.222007 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:46:35.271310 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:35.271283 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:35.271539 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:35.271522 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:35.272379 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:35.272362 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:46:47.549494 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.549434 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" podUID="26b961ed-a12f-4a67-80c9-d41e525e38f8" containerName="main" containerID="cri-o://8a9871d9dc44094ef997c78e252e57f99b57170fd5013021c5b05f186cfecca5" gracePeriod=45 Nov 04 05:46:47.622959 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.622927 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" podUID="26b961ed-a12f-4a67-80c9-d41e525e38f8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": dial tcp 10.132.0.66:8000: connect: connection refused" Nov 04 05:46:47.799128 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.799102 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:46:47.899230 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.899168 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-dshm\") pod \"26b961ed-a12f-4a67-80c9-d41e525e38f8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " Nov 04 05:46:47.899230 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.899213 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-home\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-home\") pod \"26b961ed-a12f-4a67-80c9-d41e525e38f8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " Nov 04 05:46:47.899400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.899243 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-model-cache\") pod \"26b961ed-a12f-4a67-80c9-d41e525e38f8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " Nov 04 05:46:47.899400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.899274 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/26b961ed-a12f-4a67-80c9-d41e525e38f8-tls-certs\" (UniqueName: \"kubernetes.io/secret/26b961ed-a12f-4a67-80c9-d41e525e38f8-tls-certs\") pod \"26b961ed-a12f-4a67-80c9-d41e525e38f8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " Nov 04 05:46:47.899400 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.899314 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-tmp-dir\") pod \"26b961ed-a12f-4a67-80c9-d41e525e38f8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " Nov 04 05:46:47.899548 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.899418 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26b961ed-a12f-4a67-80c9-d41e525e38f8-kube-api-access-nmmmp\" (UniqueName: \"kubernetes.io/projected/26b961ed-a12f-4a67-80c9-d41e525e38f8-kube-api-access-nmmmp\") pod \"26b961ed-a12f-4a67-80c9-d41e525e38f8\" (UID: \"26b961ed-a12f-4a67-80c9-d41e525e38f8\") " Nov 04 05:46:47.899707 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.899629 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-model-cache" pod "26b961ed-a12f-4a67-80c9-d41e525e38f8" (UID: "26b961ed-a12f-4a67-80c9-d41e525e38f8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:47.899707 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.899652 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-home" pod "26b961ed-a12f-4a67-80c9-d41e525e38f8" (UID: "26b961ed-a12f-4a67-80c9-d41e525e38f8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:47.899862 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.899830 4859 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-home\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:47.899862 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.899849 4859 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-model-cache\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:47.900045 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.900019 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-tmp-dir" pod "26b961ed-a12f-4a67-80c9-d41e525e38f8" (UID: "26b961ed-a12f-4a67-80c9-d41e525e38f8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:47.901522 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.901495 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-dshm" pod "26b961ed-a12f-4a67-80c9-d41e525e38f8" (UID: "26b961ed-a12f-4a67-80c9-d41e525e38f8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:47.901608 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.901536 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26b961ed-a12f-4a67-80c9-d41e525e38f8-kube-api-access-nmmmp" pod "26b961ed-a12f-4a67-80c9-d41e525e38f8" (UID: "26b961ed-a12f-4a67-80c9-d41e525e38f8"). InnerVolumeSpecName "kube-api-access-nmmmp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:47.901732 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:47.901697 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26b961ed-a12f-4a67-80c9-d41e525e38f8-tls-certs" pod "26b961ed-a12f-4a67-80c9-d41e525e38f8" (UID: "26b961ed-a12f-4a67-80c9-d41e525e38f8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:46:48.001273 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:48.001236 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmmmp\" (UniqueName: \"kubernetes.io/projected/26b961ed-a12f-4a67-80c9-d41e525e38f8-kube-api-access-nmmmp\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.001273 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:48.001272 4859 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-dshm\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.001273 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:48.001283 4859 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26b961ed-a12f-4a67-80c9-d41e525e38f8-tls-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.001566 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:48.001311 4859 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26b961ed-a12f-4a67-80c9-d41e525e38f8-tmp-dir\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.625515 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:48.625480 4859 generic.go:356] "Generic (PLEG): container finished" podID="26b961ed-a12f-4a67-80c9-d41e525e38f8" containerID="8a9871d9dc44094ef997c78e252e57f99b57170fd5013021c5b05f186cfecca5" exitCode=0 Nov 04 05:46:48.625996 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:48.625523 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" Nov 04 05:46:48.625996 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:48.625570 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" event={"ID":"26b961ed-a12f-4a67-80c9-d41e525e38f8","Type":"ContainerDied","Data":"8a9871d9dc44094ef997c78e252e57f99b57170fd5013021c5b05f186cfecca5"} Nov 04 05:46:48.625996 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:48.625615 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8" event={"ID":"26b961ed-a12f-4a67-80c9-d41e525e38f8","Type":"ContainerDied","Data":"4cde191106a1c0b4498e83d2e76c9a1b4c26aa9b2689862cae5fa3d81c578b50"} Nov 04 05:46:48.625996 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:48.625636 4859 scope.go:122] "RemoveContainer" containerID="8a9871d9dc44094ef997c78e252e57f99b57170fd5013021c5b05f186cfecca5" Nov 04 05:46:48.634613 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:48.634596 4859 scope.go:122] "RemoveContainer" containerID="8a9871d9dc44094ef997c78e252e57f99b57170fd5013021c5b05f186cfecca5" Nov 04 05:46:48.634917 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:46:48.634898 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a9871d9dc44094ef997c78e252e57f99b57170fd5013021c5b05f186cfecca5\": container with ID starting with 8a9871d9dc44094ef997c78e252e57f99b57170fd5013021c5b05f186cfecca5 not found: ID does not exist" containerID="8a9871d9dc44094ef997c78e252e57f99b57170fd5013021c5b05f186cfecca5" Nov 04 05:46:48.634977 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:48.634927 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a9871d9dc44094ef997c78e252e57f99b57170fd5013021c5b05f186cfecca5"} err="failed to get container status \"8a9871d9dc44094ef997c78e252e57f99b57170fd5013021c5b05f186cfecca5\": rpc error: code = NotFound desc = could not find container \"8a9871d9dc44094ef997c78e252e57f99b57170fd5013021c5b05f186cfecca5\": container with ID starting with 8a9871d9dc44094ef997c78e252e57f99b57170fd5013021c5b05f186cfecca5 not found: ID does not exist" Nov 04 05:46:48.805935 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:48.805906 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8"] Nov 04 05:46:48.860128 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:48.860100 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-961485/distinct-w2-961485-kserve-dbf9df886-2z9h8"] Nov 04 05:46:49.258617 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:49.258586 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8"] Nov 04 05:46:49.374315 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:49.374282 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f"] Nov 04 05:46:50.146447 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:50.146411 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="26b961ed-a12f-4a67-80c9-d41e525e38f8" path="/var/lib/kubelet/pods/26b961ed-a12f-4a67-80c9-d41e525e38f8/volumes" Nov 04 05:46:51.494824 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:51.494789 4859 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 05:46:51.495305 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:51.495263 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" podUID="8018ad9e-115b-47b3-9475-e6b06e8de89f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 05:46:54.259706 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.259634 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" podUID="6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0" containerName="payload-processing" containerID="cri-o://c6f4dfca622bb2bdd2a087412a33a01f51dfaf9eaa2f550e0de8a94d75050029" gracePeriod=25 Nov 04 05:46:54.374660 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.374622 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" podUID="258acbd4-6cde-4fc3-b34a-da6e04cdb963" containerName="payload-pre-processing" containerID="cri-o://32b3e484d189ce9728e35d91cd4b8cf8411bf2462da62a9a2bd44479014ecbc6" gracePeriod=25 Nov 04 05:46:54.517584 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.517527 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" Nov 04 05:46:54.616600 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.616577 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" Nov 04 05:46:54.642224 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.642202 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0-plugins-config-volume\") pod \"6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0\" (UID: \"6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0\") " Nov 04 05:46:54.642342 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.642329 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0-kube-api-access-gh4nc\" (UniqueName: \"kubernetes.io/projected/6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0-kube-api-access-gh4nc\") pod \"6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0\" (UID: \"6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0\") " Nov 04 05:46:54.642608 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.642583 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0-plugins-config-volume" pod "6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0" (UID: "6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:54.642752 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.642727 4859 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0-plugins-config-volume\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.644967 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.644947 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0-kube-api-access-gh4nc" pod "6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0" (UID: "6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0"). InnerVolumeSpecName "kube-api-access-gh4nc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:54.706132 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.706102 4859 generic.go:356] "Generic (PLEG): container finished" podID="258acbd4-6cde-4fc3-b34a-da6e04cdb963" containerID="32b3e484d189ce9728e35d91cd4b8cf8411bf2462da62a9a2bd44479014ecbc6" exitCode=0 Nov 04 05:46:54.706254 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.706153 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" Nov 04 05:46:54.706254 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.706182 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" event={"ID":"258acbd4-6cde-4fc3-b34a-da6e04cdb963","Type":"ContainerDied","Data":"32b3e484d189ce9728e35d91cd4b8cf8411bf2462da62a9a2bd44479014ecbc6"} Nov 04 05:46:54.706254 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.706238 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f" event={"ID":"258acbd4-6cde-4fc3-b34a-da6e04cdb963","Type":"ContainerDied","Data":"a1329ef0071d5164b1542d7935a94491c2ea8b91713f826e7a46562eea6c75d9"} Nov 04 05:46:54.706427 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.706262 4859 scope.go:122] "RemoveContainer" containerID="32b3e484d189ce9728e35d91cd4b8cf8411bf2462da62a9a2bd44479014ecbc6" Nov 04 05:46:54.707456 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.707435 4859 generic.go:356] "Generic (PLEG): container finished" podID="6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0" containerID="c6f4dfca622bb2bdd2a087412a33a01f51dfaf9eaa2f550e0de8a94d75050029" exitCode=0 Nov 04 05:46:54.707565 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.707470 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" Nov 04 05:46:54.707565 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.707502 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" event={"ID":"6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0","Type":"ContainerDied","Data":"c6f4dfca622bb2bdd2a087412a33a01f51dfaf9eaa2f550e0de8a94d75050029"} Nov 04 05:46:54.707565 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.707530 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8" event={"ID":"6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0","Type":"ContainerDied","Data":"789f86c4af560671fa1bb205220fd3fcb18a97868e2a084f33b1de9b66810778"} Nov 04 05:46:54.715094 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.714986 4859 scope.go:122] "RemoveContainer" containerID="32b3e484d189ce9728e35d91cd4b8cf8411bf2462da62a9a2bd44479014ecbc6" Nov 04 05:46:54.715278 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:46:54.715257 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32b3e484d189ce9728e35d91cd4b8cf8411bf2462da62a9a2bd44479014ecbc6\": container with ID starting with 32b3e484d189ce9728e35d91cd4b8cf8411bf2462da62a9a2bd44479014ecbc6 not found: ID does not exist" containerID="32b3e484d189ce9728e35d91cd4b8cf8411bf2462da62a9a2bd44479014ecbc6" Nov 04 05:46:54.715369 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.715286 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32b3e484d189ce9728e35d91cd4b8cf8411bf2462da62a9a2bd44479014ecbc6"} err="failed to get container status \"32b3e484d189ce9728e35d91cd4b8cf8411bf2462da62a9a2bd44479014ecbc6\": rpc error: code = NotFound desc = could not find container \"32b3e484d189ce9728e35d91cd4b8cf8411bf2462da62a9a2bd44479014ecbc6\": container with ID starting with 32b3e484d189ce9728e35d91cd4b8cf8411bf2462da62a9a2bd44479014ecbc6 not found: ID does not exist" Nov 04 05:46:54.715369 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.715311 4859 scope.go:122] "RemoveContainer" containerID="c6f4dfca622bb2bdd2a087412a33a01f51dfaf9eaa2f550e0de8a94d75050029" Nov 04 05:46:54.723210 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.723192 4859 scope.go:122] "RemoveContainer" containerID="c6f4dfca622bb2bdd2a087412a33a01f51dfaf9eaa2f550e0de8a94d75050029" Nov 04 05:46:54.723456 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:46:54.723439 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6f4dfca622bb2bdd2a087412a33a01f51dfaf9eaa2f550e0de8a94d75050029\": container with ID starting with c6f4dfca622bb2bdd2a087412a33a01f51dfaf9eaa2f550e0de8a94d75050029 not found: ID does not exist" containerID="c6f4dfca622bb2bdd2a087412a33a01f51dfaf9eaa2f550e0de8a94d75050029" Nov 04 05:46:54.723504 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.723462 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6f4dfca622bb2bdd2a087412a33a01f51dfaf9eaa2f550e0de8a94d75050029"} err="failed to get container status \"c6f4dfca622bb2bdd2a087412a33a01f51dfaf9eaa2f550e0de8a94d75050029\": rpc error: code = NotFound desc = could not find container \"c6f4dfca622bb2bdd2a087412a33a01f51dfaf9eaa2f550e0de8a94d75050029\": container with ID starting with c6f4dfca622bb2bdd2a087412a33a01f51dfaf9eaa2f550e0de8a94d75050029 not found: ID does not exist" Nov 04 05:46:54.743361 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.743331 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/258acbd4-6cde-4fc3-b34a-da6e04cdb963-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/258acbd4-6cde-4fc3-b34a-da6e04cdb963-plugins-config-volume\") pod \"258acbd4-6cde-4fc3-b34a-da6e04cdb963\" (UID: \"258acbd4-6cde-4fc3-b34a-da6e04cdb963\") " Nov 04 05:46:54.743449 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.743398 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/258acbd4-6cde-4fc3-b34a-da6e04cdb963-kube-api-access-bw4xk\" (UniqueName: \"kubernetes.io/projected/258acbd4-6cde-4fc3-b34a-da6e04cdb963-kube-api-access-bw4xk\") pod \"258acbd4-6cde-4fc3-b34a-da6e04cdb963\" (UID: \"258acbd4-6cde-4fc3-b34a-da6e04cdb963\") " Nov 04 05:46:54.743728 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.743703 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/258acbd4-6cde-4fc3-b34a-da6e04cdb963-plugins-config-volume" pod "258acbd4-6cde-4fc3-b34a-da6e04cdb963" (UID: "258acbd4-6cde-4fc3-b34a-da6e04cdb963"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:54.743886 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.743860 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gh4nc\" (UniqueName: \"kubernetes.io/projected/6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0-kube-api-access-gh4nc\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.743886 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.743882 4859 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/258acbd4-6cde-4fc3-b34a-da6e04cdb963-plugins-config-volume\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.745367 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.745342 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/258acbd4-6cde-4fc3-b34a-da6e04cdb963-kube-api-access-bw4xk" pod "258acbd4-6cde-4fc3-b34a-da6e04cdb963" (UID: "258acbd4-6cde-4fc3-b34a-da6e04cdb963"). InnerVolumeSpecName "kube-api-access-bw4xk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:54.844755 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.844658 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw4xk\" (UniqueName: \"kubernetes.io/projected/258acbd4-6cde-4fc3-b34a-da6e04cdb963-kube-api-access-bw4xk\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.943753 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.943723 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8"] Nov 04 05:46:54.948440 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:54.948418 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-961485-78dd6864c6-mlkj8"] Nov 04 05:46:55.136956 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:55.136857 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f"] Nov 04 05:46:55.139036 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:55.139013 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-961485-5d688b9bc-hg85f"] Nov 04 05:46:56.119932 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:56.116979 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="258acbd4-6cde-4fc3-b34a-da6e04cdb963" path="/var/lib/kubelet/pods/258acbd4-6cde-4fc3-b34a-da6e04cdb963/volumes" Nov 04 05:46:56.119932 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:46:56.117578 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0" path="/var/lib/kubelet/pods/6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0/volumes" Nov 04 05:46:58.341063 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:46:58.341028 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:47:04.823471 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:04.823432 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2"] Nov 04 05:47:04.824032 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:04.823717 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" podUID="f952eaf6-6354-4094-9a30-3c032183dfed" containerName="maas-api" containerID="cri-o://c5e1422673fe31a827e18c3221b6749e00f611d48204825eda30a78665682253" gracePeriod=30 Nov 04 05:47:06.088849 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.088822 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:47:06.093246 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.093225 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p"] Nov 04 05:47:06.093436 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.093416 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" podUID="e6446e2c-33aa-434c-b167-8c69862baaa9" containerName="maas-api" containerID="cri-o://f5b47c57a3e13ab88bd110f3c07d92120e832bb7de303721747eb60516ead368" gracePeriod=30 Nov 04 05:47:06.255079 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.254958 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-maas-api-tls\") pod \"f952eaf6-6354-4094-9a30-3c032183dfed\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " Nov 04 05:47:06.255257 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.255097 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f952eaf6-6354-4094-9a30-3c032183dfed-kube-api-access-6wzcv\" (UniqueName: \"kubernetes.io/projected/f952eaf6-6354-4094-9a30-3c032183dfed-kube-api-access-6wzcv\") pod \"f952eaf6-6354-4094-9a30-3c032183dfed\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " Nov 04 05:47:06.255257 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.255131 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f952eaf6-6354-4094-9a30-3c032183dfed-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f952eaf6-6354-4094-9a30-3c032183dfed-openshift-service-ca\") pod \"f952eaf6-6354-4094-9a30-3c032183dfed\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " Nov 04 05:47:06.255257 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.255177 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-metrics-certs\") pod \"f952eaf6-6354-4094-9a30-3c032183dfed\" (UID: \"f952eaf6-6354-4094-9a30-3c032183dfed\") " Nov 04 05:47:06.255847 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.255795 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f952eaf6-6354-4094-9a30-3c032183dfed-openshift-service-ca" pod "f952eaf6-6354-4094-9a30-3c032183dfed" (UID: "f952eaf6-6354-4094-9a30-3c032183dfed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:06.257607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.257571 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f952eaf6-6354-4094-9a30-3c032183dfed-kube-api-access-6wzcv" pod "f952eaf6-6354-4094-9a30-3c032183dfed" (UID: "f952eaf6-6354-4094-9a30-3c032183dfed"). InnerVolumeSpecName "kube-api-access-6wzcv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:06.257813 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.257698 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-metrics-certs" pod "f952eaf6-6354-4094-9a30-3c032183dfed" (UID: "f952eaf6-6354-4094-9a30-3c032183dfed"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:47:06.258112 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.258089 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-maas-api-tls" pod "f952eaf6-6354-4094-9a30-3c032183dfed" (UID: "f952eaf6-6354-4094-9a30-3c032183dfed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:47:06.360647 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.358277 4859 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-maas-api-tls\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:06.360647 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.358318 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wzcv\" (UniqueName: \"kubernetes.io/projected/f952eaf6-6354-4094-9a30-3c032183dfed-kube-api-access-6wzcv\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:06.360647 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.358335 4859 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f952eaf6-6354-4094-9a30-3c032183dfed-openshift-service-ca\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:06.360647 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.358349 4859 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f952eaf6-6354-4094-9a30-3c032183dfed-metrics-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:06.369800 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.369777 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:47:06.559887 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.559805 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-maas-api-tls\") pod \"e6446e2c-33aa-434c-b167-8c69862baaa9\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " Nov 04 05:47:06.560060 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.559892 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-metrics-certs\") pod \"e6446e2c-33aa-434c-b167-8c69862baaa9\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " Nov 04 05:47:06.560060 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.559997 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6446e2c-33aa-434c-b167-8c69862baaa9-kube-api-access-xrnfh\" (UniqueName: \"kubernetes.io/projected/e6446e2c-33aa-434c-b167-8c69862baaa9-kube-api-access-xrnfh\") pod \"e6446e2c-33aa-434c-b167-8c69862baaa9\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " Nov 04 05:47:06.560060 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.560021 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6446e2c-33aa-434c-b167-8c69862baaa9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6446e2c-33aa-434c-b167-8c69862baaa9-openshift-service-ca\") pod \"e6446e2c-33aa-434c-b167-8c69862baaa9\" (UID: \"e6446e2c-33aa-434c-b167-8c69862baaa9\") " Nov 04 05:47:06.560447 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.560421 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6446e2c-33aa-434c-b167-8c69862baaa9-openshift-service-ca" pod "e6446e2c-33aa-434c-b167-8c69862baaa9" (UID: "e6446e2c-33aa-434c-b167-8c69862baaa9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:06.562169 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.562136 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-metrics-certs" pod "e6446e2c-33aa-434c-b167-8c69862baaa9" (UID: "e6446e2c-33aa-434c-b167-8c69862baaa9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:47:06.562169 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.562136 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6446e2c-33aa-434c-b167-8c69862baaa9-kube-api-access-xrnfh" pod "e6446e2c-33aa-434c-b167-8c69862baaa9" (UID: "e6446e2c-33aa-434c-b167-8c69862baaa9"). InnerVolumeSpecName "kube-api-access-xrnfh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:06.562307 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.562173 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-maas-api-tls" pod "e6446e2c-33aa-434c-b167-8c69862baaa9" (UID: "e6446e2c-33aa-434c-b167-8c69862baaa9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:47:06.660965 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.660927 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrnfh\" (UniqueName: \"kubernetes.io/projected/e6446e2c-33aa-434c-b167-8c69862baaa9-kube-api-access-xrnfh\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:06.660965 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.660957 4859 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6446e2c-33aa-434c-b167-8c69862baaa9-openshift-service-ca\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:06.660965 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.660968 4859 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-maas-api-tls\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:06.660965 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.660977 4859 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6446e2c-33aa-434c-b167-8c69862baaa9-metrics-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:06.769716 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.769679 4859 generic.go:356] "Generic (PLEG): container finished" podID="e6446e2c-33aa-434c-b167-8c69862baaa9" containerID="f5b47c57a3e13ab88bd110f3c07d92120e832bb7de303721747eb60516ead368" exitCode=0 Nov 04 05:47:06.769894 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.769742 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" Nov 04 05:47:06.769894 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.769764 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" event={"ID":"e6446e2c-33aa-434c-b167-8c69862baaa9","Type":"ContainerDied","Data":"f5b47c57a3e13ab88bd110f3c07d92120e832bb7de303721747eb60516ead368"} Nov 04 05:47:06.769894 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.769817 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p" event={"ID":"e6446e2c-33aa-434c-b167-8c69862baaa9","Type":"ContainerDied","Data":"0dcfeb2c30842ad69040a0cbc02fd9eb31ad9e68aeb064b8732ccf2994a4be63"} Nov 04 05:47:06.769894 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.769839 4859 scope.go:122] "RemoveContainer" containerID="f5b47c57a3e13ab88bd110f3c07d92120e832bb7de303721747eb60516ead368" Nov 04 05:47:06.771225 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.771200 4859 generic.go:356] "Generic (PLEG): container finished" podID="f952eaf6-6354-4094-9a30-3c032183dfed" containerID="c5e1422673fe31a827e18c3221b6749e00f611d48204825eda30a78665682253" exitCode=0 Nov 04 05:47:06.771355 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.771231 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" event={"ID":"f952eaf6-6354-4094-9a30-3c032183dfed","Type":"ContainerDied","Data":"c5e1422673fe31a827e18c3221b6749e00f611d48204825eda30a78665682253"} Nov 04 05:47:06.771355 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.771246 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" Nov 04 05:47:06.771355 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.771280 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2" event={"ID":"f952eaf6-6354-4094-9a30-3c032183dfed","Type":"ContainerDied","Data":"1b8ccca54a84819c8c427862835c0995efcbef4a43348d5f0805fda5d11c7bb2"} Nov 04 05:47:06.779707 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.779653 4859 scope.go:122] "RemoveContainer" containerID="f5b47c57a3e13ab88bd110f3c07d92120e832bb7de303721747eb60516ead368" Nov 04 05:47:06.780065 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:47:06.780036 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5b47c57a3e13ab88bd110f3c07d92120e832bb7de303721747eb60516ead368\": container with ID starting with f5b47c57a3e13ab88bd110f3c07d92120e832bb7de303721747eb60516ead368 not found: ID does not exist" containerID="f5b47c57a3e13ab88bd110f3c07d92120e832bb7de303721747eb60516ead368" Nov 04 05:47:06.780156 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.780063 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5b47c57a3e13ab88bd110f3c07d92120e832bb7de303721747eb60516ead368"} err="failed to get container status \"f5b47c57a3e13ab88bd110f3c07d92120e832bb7de303721747eb60516ead368\": rpc error: code = NotFound desc = could not find container \"f5b47c57a3e13ab88bd110f3c07d92120e832bb7de303721747eb60516ead368\": container with ID starting with f5b47c57a3e13ab88bd110f3c07d92120e832bb7de303721747eb60516ead368 not found: ID does not exist" Nov 04 05:47:06.780156 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.780079 4859 scope.go:122] "RemoveContainer" containerID="c5e1422673fe31a827e18c3221b6749e00f611d48204825eda30a78665682253" Nov 04 05:47:06.788545 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.788527 4859 scope.go:122] "RemoveContainer" containerID="c5e1422673fe31a827e18c3221b6749e00f611d48204825eda30a78665682253" Nov 04 05:47:06.788809 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:47:06.788787 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5e1422673fe31a827e18c3221b6749e00f611d48204825eda30a78665682253\": container with ID starting with c5e1422673fe31a827e18c3221b6749e00f611d48204825eda30a78665682253 not found: ID does not exist" containerID="c5e1422673fe31a827e18c3221b6749e00f611d48204825eda30a78665682253" Nov 04 05:47:06.788867 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.788820 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5e1422673fe31a827e18c3221b6749e00f611d48204825eda30a78665682253"} err="failed to get container status \"c5e1422673fe31a827e18c3221b6749e00f611d48204825eda30a78665682253\": rpc error: code = NotFound desc = could not find container \"c5e1422673fe31a827e18c3221b6749e00f611d48204825eda30a78665682253\": container with ID starting with c5e1422673fe31a827e18c3221b6749e00f611d48204825eda30a78665682253 not found: ID does not exist" Nov 04 05:47:06.953959 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.953928 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p"] Nov 04 05:47:06.959260 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:06.959237 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-901a00-59d87b6dd8-5wm6p"] Nov 04 05:47:07.077528 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:07.077496 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2"] Nov 04 05:47:07.084069 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:07.084047 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-961485-5d6988956d-crlb2"] Nov 04 05:47:08.115086 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:08.115055 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6446e2c-33aa-434c-b167-8c69862baaa9" path="/var/lib/kubelet/pods/e6446e2c-33aa-434c-b167-8c69862baaa9/volumes" Nov 04 05:47:08.115483 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:08.115469 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f952eaf6-6354-4094-9a30-3c032183dfed" path="/var/lib/kubelet/pods/f952eaf6-6354-4094-9a30-3c032183dfed/volumes" Nov 04 05:47:10.232807 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:10.232780 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/2.log" Nov 04 05:47:10.237411 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:10.237389 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-qnfh8_58bf7d20-81e0-4dbc-85f2-32ded8f14e5f/console-operator/2.log" Nov 04 05:47:12.309679 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.309638 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7"] Nov 04 05:47:12.310075 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.310061 4859 cpu_manager.go:414] "containerMap: removing container" podUID="26b961ed-a12f-4a67-80c9-d41e525e38f8" containerName="main" Nov 04 05:47:12.310118 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.310075 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26b961ed-a12f-4a67-80c9-d41e525e38f8" containerName="main" Nov 04 05:47:12.310118 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.310094 4859 cpu_manager.go:414] "containerMap: removing container" podUID="6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0" containerName="payload-processing" Nov 04 05:47:12.310118 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.310099 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0" containerName="payload-processing" Nov 04 05:47:12.310209 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.310120 4859 cpu_manager.go:414] "containerMap: removing container" podUID="e6446e2c-33aa-434c-b167-8c69862baaa9" containerName="maas-api" Nov 04 05:47:12.310209 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.310126 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6446e2c-33aa-434c-b167-8c69862baaa9" containerName="maas-api" Nov 04 05:47:12.310209 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.310134 4859 cpu_manager.go:414] "containerMap: removing container" podUID="258acbd4-6cde-4fc3-b34a-da6e04cdb963" containerName="payload-pre-processing" Nov 04 05:47:12.310209 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.310138 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="258acbd4-6cde-4fc3-b34a-da6e04cdb963" containerName="payload-pre-processing" Nov 04 05:47:12.310209 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.310148 4859 cpu_manager.go:414] "containerMap: removing container" podUID="f952eaf6-6354-4094-9a30-3c032183dfed" containerName="maas-api" Nov 04 05:47:12.310209 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.310152 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f952eaf6-6354-4094-9a30-3c032183dfed" containerName="maas-api" Nov 04 05:47:12.310384 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.310237 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="26b961ed-a12f-4a67-80c9-d41e525e38f8" containerName="main" Nov 04 05:47:12.310384 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.310246 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f9bc5b2-a5c8-47bd-a478-d40f2e11ccf0" containerName="payload-processing" Nov 04 05:47:12.310384 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.310253 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="258acbd4-6cde-4fc3-b34a-da6e04cdb963" containerName="payload-pre-processing" Nov 04 05:47:12.310384 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.310261 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6446e2c-33aa-434c-b167-8c69862baaa9" containerName="maas-api" Nov 04 05:47:12.310384 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.310267 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="f952eaf6-6354-4094-9a30-3c032183dfed" containerName="maas-api" Nov 04 05:47:12.315316 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.315296 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.418457 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.418428 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.418618 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.418479 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.418618 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.418518 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qblw\" (UniqueName: \"kubernetes.io/projected/3d2b29d5-afab-4bfd-878f-615e74d45cfa-kube-api-access-2qblw\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.418618 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.418548 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.418808 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.418720 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.418808 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.418785 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.418875 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.418811 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.418875 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.418838 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.418946 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.418895 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.520157 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.520126 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.520348 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.520167 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qblw\" (UniqueName: \"kubernetes.io/projected/3d2b29d5-afab-4bfd-878f-615e74d45cfa-kube-api-access-2qblw\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.520348 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.520193 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.520348 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.520238 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.520348 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.520278 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.520348 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.520309 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.520348 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.520339 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.520718 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.520393 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.520718 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.520474 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.520718 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.520620 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.520877 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.520737 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.520877 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.520760 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.520877 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.520816 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.521242 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.521216 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.522996 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.522974 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.523162 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.523146 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.549441 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.549413 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7"] Nov 04 05:47:12.585799 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.585728 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.585908 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.585882 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qblw\" (UniqueName: \"kubernetes.io/projected/3d2b29d5-afab-4bfd-878f-615e74d45cfa-kube-api-access-2qblw\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.630132 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.630098 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:12.663184 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.663153 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484"] Nov 04 05:47:12.668440 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.668416 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.703331 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.703295 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9"] Nov 04 05:47:12.713073 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.713036 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.725328 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.725298 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.725538 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.725515 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.725645 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.725632 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6f3c605-6d79-4057-9203-c965c9084fbe-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.725788 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.725774 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.725916 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.725904 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.726111 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.726094 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.726253 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.726237 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.726358 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.726345 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.726478 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.726463 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snpmx\" (UniqueName: \"kubernetes.io/projected/a6f3c605-6d79-4057-9203-c965c9084fbe-kube-api-access-snpmx\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.735905 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.735883 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484"] Nov 04 05:47:12.759725 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.759615 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9"] Nov 04 05:47:12.827796 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.827765 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snpmx\" (UniqueName: \"kubernetes.io/projected/a6f3c605-6d79-4057-9203-c965c9084fbe-kube-api-access-snpmx\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.827980 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.827805 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.827980 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.827831 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.827980 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.827884 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6f3c605-6d79-4057-9203-c965c9084fbe-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.827980 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.827934 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.828213 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828093 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.828213 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828169 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.828327 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828227 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.828327 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828254 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1987453d-4e27-42c8-9737-a6689b817176-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.828327 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828267 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.828327 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828304 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.828574 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828336 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.828574 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828458 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.828574 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828495 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1987453d-4e27-42c8-9737-a6689b817176-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.828574 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828535 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.828806 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828579 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.828806 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828620 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1987453d-4e27-42c8-9737-a6689b817176-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.828806 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828688 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k89j\" (UniqueName: \"kubernetes.io/projected/1987453d-4e27-42c8-9737-a6689b817176-kube-api-access-8k89j\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.828806 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828723 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6f3c605-6d79-4057-9203-c965c9084fbe-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.828806 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828742 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.829060 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828881 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.829060 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.828991 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.829060 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.829050 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.830101 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.830080 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.830928 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.830905 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.905480 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.905420 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.905578 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.905522 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snpmx\" (UniqueName: \"kubernetes.io/projected/a6f3c605-6d79-4057-9203-c965c9084fbe-kube-api-access-snpmx\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:12.929502 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.929458 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.929635 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.929511 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.929635 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.929554 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1987453d-4e27-42c8-9737-a6689b817176-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.929635 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.929586 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.929635 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.929612 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.929834 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.929792 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1987453d-4e27-42c8-9737-a6689b817176-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.929888 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.929851 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.929943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.929890 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.929943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.929905 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.929943 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.929932 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1987453d-4e27-42c8-9737-a6689b817176-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.930102 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.929961 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k89j\" (UniqueName: \"kubernetes.io/projected/1987453d-4e27-42c8-9737-a6689b817176-kube-api-access-8k89j\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.930102 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.929964 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.930207 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.930061 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.930351 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.930333 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1987453d-4e27-42c8-9737-a6689b817176-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.932078 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.932054 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.932429 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.932405 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1987453d-4e27-42c8-9737-a6689b817176-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.995711 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.995678 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1987453d-4e27-42c8-9737-a6689b817176-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:12.995711 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:12.995700 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k89j\" (UniqueName: \"kubernetes.io/projected/1987453d-4e27-42c8-9737-a6689b817176-kube-api-access-8k89j\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:13.030985 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.030956 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:13.049623 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.049596 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:13.106070 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.101654 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7"] Nov 04 05:47:13.116630 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.108449 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:47:13.116630 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.108515 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:47:13.116630 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.108544 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:47:13.198492 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.197240 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484"] Nov 04 05:47:13.217105 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.217075 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s"] Nov 04 05:47:13.217288 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.217266 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" podUID="7e5b156e-c723-400a-b76e-4d914ad58712" containerName="istio-proxy" containerID="cri-o://dc42e57ca337d7f0b9ca62ec7eb62f3114d35e3db6988407e20ac7db4f1e8760" gracePeriod=30 Nov 04 05:47:13.218082 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.218057 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm"] Nov 04 05:47:13.218301 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.218245 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" podUID="8018ad9e-115b-47b3-9475-e6b06e8de89f" containerName="istio-proxy" containerID="cri-o://fd3328be4b32ed7e7db82b3f40fd95bff8c0cee82cd2e0c7f905a2d275d0b725" gracePeriod=30 Nov 04 05:47:13.219994 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.219975 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9"] Nov 04 05:47:13.445058 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.444962 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484"] Nov 04 05:47:13.450260 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:47:13.450228 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6f3c605_6d79_4057_9203_c965c9084fbe.slice/crio-219eedeef30059b29c6328042602e22c608c344455d97a0148c22d5a04ad0a01 WatchSource:0}: Error finding container 219eedeef30059b29c6328042602e22c608c344455d97a0148c22d5a04ad0a01: Status 404 returned error can't find the container with id 219eedeef30059b29c6328042602e22c608c344455d97a0148c22d5a04ad0a01 Nov 04 05:47:13.453197 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.453165 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:47:13.453290 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.453234 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:47:13.453290 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.453262 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:47:13.483893 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.483868 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9"] Nov 04 05:47:13.502399 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:47:13.502374 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1987453d_4e27_42c8_9737_a6689b817176.slice/crio-b1c3ba49f7db1121ae56ac6976fc21edc45dd6ce75535db32831bd34ec0a3d87 WatchSource:0}: Error finding container b1c3ba49f7db1121ae56ac6976fc21edc45dd6ce75535db32831bd34ec0a3d87: Status 404 returned error can't find the container with id b1c3ba49f7db1121ae56ac6976fc21edc45dd6ce75535db32831bd34ec0a3d87 Nov 04 05:47:13.504918 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.504884 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:47:13.504990 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.504964 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:47:13.505025 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.505007 4859 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 05:47:13.824233 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.824130 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" event={"ID":"a6f3c605-6d79-4057-9203-c965c9084fbe","Type":"ContainerStarted","Data":"526e4b1fdf27556791068b48accc678deeca98be4e7415346073541aacdea80d"} Nov 04 05:47:13.824233 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.824195 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" event={"ID":"a6f3c605-6d79-4057-9203-c965c9084fbe","Type":"ContainerStarted","Data":"219eedeef30059b29c6328042602e22c608c344455d97a0148c22d5a04ad0a01"} Nov 04 05:47:13.824233 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.824199 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" podUID="a6f3c605-6d79-4057-9203-c965c9084fbe" containerName="istio-proxy" containerID="cri-o://526e4b1fdf27556791068b48accc678deeca98be4e7415346073541aacdea80d" gracePeriod=30 Nov 04 05:47:13.826120 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.826090 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" event={"ID":"1987453d-4e27-42c8-9737-a6689b817176","Type":"ContainerStarted","Data":"787b80e46ead8afccf48b86abb1b8987d188a43d08a2cc02da0806d8497a2277"} Nov 04 05:47:13.826237 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.826138 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" event={"ID":"1987453d-4e27-42c8-9737-a6689b817176","Type":"ContainerStarted","Data":"b1c3ba49f7db1121ae56ac6976fc21edc45dd6ce75535db32831bd34ec0a3d87"} Nov 04 05:47:13.826237 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.826177 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" podUID="1987453d-4e27-42c8-9737-a6689b817176" containerName="istio-proxy" containerID="cri-o://787b80e46ead8afccf48b86abb1b8987d188a43d08a2cc02da0806d8497a2277" gracePeriod=30 Nov 04 05:47:13.828976 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.828944 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" event={"ID":"3d2b29d5-afab-4bfd-878f-615e74d45cfa","Type":"ContainerStarted","Data":"ceb6029c8211f6343ba8bb39490265c9df09039e47f06b20afef36bec61058e5"} Nov 04 05:47:13.829070 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.829003 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" event={"ID":"3d2b29d5-afab-4bfd-878f-615e74d45cfa","Type":"ContainerStarted","Data":"a6e01a11edec078ceaf41a83f9545e7b66da86b4a3a192b8364417706c2fd795"} Nov 04 05:47:13.971691 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:13.971629 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" podStartSLOduration=1.9716153539999999 podStartE2EDuration="1.971615354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:47:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:47:13.970476618 +0000 UTC m=+1203.972112346" watchObservedRunningTime="2026-11-04 05:47:13.971615354 +0000 UTC m=+1203.973251076" Nov 04 05:47:14.109985 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:14.109865 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" podStartSLOduration=2.109845836 podStartE2EDuration="2.109845836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:47:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:47:14.108189373 +0000 UTC m=+1204.109825119" watchObservedRunningTime="2026-11-04 05:47:14.109845836 +0000 UTC m=+1204.111481561" Nov 04 05:47:14.237799 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:14.237741 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" podStartSLOduration=2.237725662 podStartE2EDuration="2.237725662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:47:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:47:14.236378426 +0000 UTC m=+1204.238014153" watchObservedRunningTime="2026-11-04 05:47:14.237725662 +0000 UTC m=+1204.239361392" Nov 04 05:47:14.631296 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:14.631262 4859 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:14.632865 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:14.632841 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Nov 04 05:47:14.632983 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:14.632885 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" podUID="3d2b29d5-afab-4bfd-878f-615e74d45cfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Nov 04 05:47:15.630398 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:15.630363 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Nov 04 05:47:15.630600 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:15.630421 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" podUID="3d2b29d5-afab-4bfd-878f-615e74d45cfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Nov 04 05:47:16.631209 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:16.631172 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Nov 04 05:47:16.631593 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:16.631242 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" podUID="3d2b29d5-afab-4bfd-878f-615e74d45cfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Nov 04 05:47:17.630590 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:17.630554 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Nov 04 05:47:17.630810 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:17.630600 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" podUID="3d2b29d5-afab-4bfd-878f-615e74d45cfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Nov 04 05:47:18.357719 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.357544 4859 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Nov 04 05:47:18.357719 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.357607 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" podUID="7e5b156e-c723-400a-b76e-4d914ad58712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Nov 04 05:47:18.536941 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.536912 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:47:18.557801 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.557781 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:47:18.611589 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.611555 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-istio-data\") pod \"7e5b156e-c723-400a-b76e-4d914ad58712\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " Nov 04 05:47:18.611783 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.611612 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-workload-certs\") pod \"8018ad9e-115b-47b3-9475-e6b06e8de89f\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " Nov 04 05:47:18.611783 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.611641 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-credential-socket\") pod \"7e5b156e-c723-400a-b76e-4d914ad58712\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " Nov 04 05:47:18.611783 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.611704 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-token\" (UniqueName: \"kubernetes.io/projected/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-token\") pod \"8018ad9e-115b-47b3-9475-e6b06e8de89f\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " Nov 04 05:47:18.611783 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.611741 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e5b156e-c723-400a-b76e-4d914ad58712-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e5b156e-c723-400a-b76e-4d914ad58712-istiod-ca-cert\") pod \"7e5b156e-c723-400a-b76e-4d914ad58712\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " Nov 04 05:47:18.612007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.611803 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7e5b156e-c723-400a-b76e-4d914ad58712-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e5b156e-c723-400a-b76e-4d914ad58712-istio-podinfo\") pod \"7e5b156e-c723-400a-b76e-4d914ad58712\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " Nov 04 05:47:18.612007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.611874 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-workload-certs" pod "8018ad9e-115b-47b3-9475-e6b06e8de89f" (UID: "8018ad9e-115b-47b3-9475-e6b06e8de89f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.612007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.611871 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-istio-data" pod "7e5b156e-c723-400a-b76e-4d914ad58712" (UID: "7e5b156e-c723-400a-b76e-4d914ad58712"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.612007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.611901 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8018ad9e-115b-47b3-9475-e6b06e8de89f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8018ad9e-115b-47b3-9475-e6b06e8de89f-istiod-ca-cert\") pod \"8018ad9e-115b-47b3-9475-e6b06e8de89f\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " Nov 04 05:47:18.612007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.611932 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-workload-socket\") pod \"8018ad9e-115b-47b3-9475-e6b06e8de89f\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " Nov 04 05:47:18.612007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.611968 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e5b156e-c723-400a-b76e-4d914ad58712-kube-api-access-jkjg7\" (UniqueName: \"kubernetes.io/projected/7e5b156e-c723-400a-b76e-4d914ad58712-kube-api-access-jkjg7\") pod \"7e5b156e-c723-400a-b76e-4d914ad58712\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " Nov 04 05:47:18.612007 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.611995 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-podinfo\") pod \"8018ad9e-115b-47b3-9475-e6b06e8de89f\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " Nov 04 05:47:18.612326 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612031 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-workload-socket\") pod \"7e5b156e-c723-400a-b76e-4d914ad58712\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " Nov 04 05:47:18.612326 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612060 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-workload-certs\") pod \"7e5b156e-c723-400a-b76e-4d914ad58712\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " Nov 04 05:47:18.612326 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612084 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-istio-envoy\") pod \"7e5b156e-c723-400a-b76e-4d914ad58712\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " Nov 04 05:47:18.612326 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612116 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-data\") pod \"8018ad9e-115b-47b3-9475-e6b06e8de89f\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " Nov 04 05:47:18.612326 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612150 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8018ad9e-115b-47b3-9475-e6b06e8de89f-kube-api-access-cqdkp\" (UniqueName: \"kubernetes.io/projected/8018ad9e-115b-47b3-9475-e6b06e8de89f-kube-api-access-cqdkp\") pod \"8018ad9e-115b-47b3-9475-e6b06e8de89f\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " Nov 04 05:47:18.612326 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612161 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e5b156e-c723-400a-b76e-4d914ad58712-istiod-ca-cert" pod "7e5b156e-c723-400a-b76e-4d914ad58712" (UID: "7e5b156e-c723-400a-b76e-4d914ad58712"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:18.612326 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612177 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-credential-socket\") pod \"8018ad9e-115b-47b3-9475-e6b06e8de89f\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " Nov 04 05:47:18.612326 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612210 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e5b156e-c723-400a-b76e-4d914ad58712-istio-token\" (UniqueName: \"kubernetes.io/projected/7e5b156e-c723-400a-b76e-4d914ad58712-istio-token\") pod \"7e5b156e-c723-400a-b76e-4d914ad58712\" (UID: \"7e5b156e-c723-400a-b76e-4d914ad58712\") " Nov 04 05:47:18.612326 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612234 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-envoy\") pod \"8018ad9e-115b-47b3-9475-e6b06e8de89f\" (UID: \"8018ad9e-115b-47b3-9475-e6b06e8de89f\") " Nov 04 05:47:18.612326 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612304 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-workload-socket" pod "7e5b156e-c723-400a-b76e-4d914ad58712" (UID: "7e5b156e-c723-400a-b76e-4d914ad58712"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.612841 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612491 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8018ad9e-115b-47b3-9475-e6b06e8de89f-istiod-ca-cert" pod "8018ad9e-115b-47b3-9475-e6b06e8de89f" (UID: "8018ad9e-115b-47b3-9475-e6b06e8de89f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:18.612841 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612532 4859 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-istio-data\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.612841 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612544 4859 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-workload-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.612841 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612553 4859 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e5b156e-c723-400a-b76e-4d914ad58712-istiod-ca-cert\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.612841 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612563 4859 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-workload-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.612841 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612593 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-workload-socket" pod "8018ad9e-115b-47b3-9475-e6b06e8de89f" (UID: "8018ad9e-115b-47b3-9475-e6b06e8de89f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.612841 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.612813 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-workload-certs" pod "7e5b156e-c723-400a-b76e-4d914ad58712" (UID: "7e5b156e-c723-400a-b76e-4d914ad58712"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.613200 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.613041 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-credential-socket" pod "7e5b156e-c723-400a-b76e-4d914ad58712" (UID: "7e5b156e-c723-400a-b76e-4d914ad58712"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.613263 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.613224 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-credential-socket" pod "8018ad9e-115b-47b3-9475-e6b06e8de89f" (UID: "8018ad9e-115b-47b3-9475-e6b06e8de89f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.613402 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.613382 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-data" pod "8018ad9e-115b-47b3-9475-e6b06e8de89f" (UID: "8018ad9e-115b-47b3-9475-e6b06e8de89f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.614327 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.614284 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-token" pod "8018ad9e-115b-47b3-9475-e6b06e8de89f" (UID: "8018ad9e-115b-47b3-9475-e6b06e8de89f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:18.614690 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.614632 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e5b156e-c723-400a-b76e-4d914ad58712-istio-podinfo" pod "7e5b156e-c723-400a-b76e-4d914ad58712" (UID: "7e5b156e-c723-400a-b76e-4d914ad58712"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:18.615724 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.615699 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-podinfo" pod "8018ad9e-115b-47b3-9475-e6b06e8de89f" (UID: "8018ad9e-115b-47b3-9475-e6b06e8de89f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:18.615890 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.615732 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-envoy" pod "8018ad9e-115b-47b3-9475-e6b06e8de89f" (UID: "8018ad9e-115b-47b3-9475-e6b06e8de89f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.616075 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.616056 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8018ad9e-115b-47b3-9475-e6b06e8de89f-kube-api-access-cqdkp" pod "8018ad9e-115b-47b3-9475-e6b06e8de89f" (UID: "8018ad9e-115b-47b3-9475-e6b06e8de89f"). InnerVolumeSpecName "kube-api-access-cqdkp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:18.616934 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.616915 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-istio-envoy" pod "7e5b156e-c723-400a-b76e-4d914ad58712" (UID: "7e5b156e-c723-400a-b76e-4d914ad58712"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.617234 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.617205 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e5b156e-c723-400a-b76e-4d914ad58712-istio-token" pod "7e5b156e-c723-400a-b76e-4d914ad58712" (UID: "7e5b156e-c723-400a-b76e-4d914ad58712"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:18.617451 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.617436 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e5b156e-c723-400a-b76e-4d914ad58712-kube-api-access-jkjg7" pod "7e5b156e-c723-400a-b76e-4d914ad58712" (UID: "7e5b156e-c723-400a-b76e-4d914ad58712"). InnerVolumeSpecName "kube-api-access-jkjg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:18.631117 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.631083 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Nov 04 05:47:18.631199 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.631151 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" podUID="3d2b29d5-afab-4bfd-878f-615e74d45cfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Nov 04 05:47:18.713458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.713342 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqdkp\" (UniqueName: \"kubernetes.io/projected/8018ad9e-115b-47b3-9475-e6b06e8de89f-kube-api-access-cqdkp\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.713458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.713367 4859 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-credential-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.713458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.713378 4859 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e5b156e-c723-400a-b76e-4d914ad58712-istio-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.713458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.713387 4859 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-envoy\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.713458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.713396 4859 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-credential-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.713458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.713404 4859 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.713458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.713413 4859 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e5b156e-c723-400a-b76e-4d914ad58712-istio-podinfo\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.713458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.713421 4859 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8018ad9e-115b-47b3-9475-e6b06e8de89f-istiod-ca-cert\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.713458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.713430 4859 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-workload-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.713458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.713438 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkjg7\" (UniqueName: \"kubernetes.io/projected/7e5b156e-c723-400a-b76e-4d914ad58712-kube-api-access-jkjg7\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.713458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.713447 4859 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-podinfo\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.713458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.713455 4859 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-workload-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.713458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.713463 4859 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e5b156e-c723-400a-b76e-4d914ad58712-istio-envoy\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.713458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.713472 4859 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8018ad9e-115b-47b3-9475-e6b06e8de89f-istio-data\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.871559 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.871525 4859 generic.go:356] "Generic (PLEG): container finished" podID="8018ad9e-115b-47b3-9475-e6b06e8de89f" containerID="fd3328be4b32ed7e7db82b3f40fd95bff8c0cee82cd2e0c7f905a2d275d0b725" exitCode=0 Nov 04 05:47:18.871741 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.871585 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" Nov 04 05:47:18.871741 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.871599 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" event={"ID":"8018ad9e-115b-47b3-9475-e6b06e8de89f","Type":"ContainerDied","Data":"fd3328be4b32ed7e7db82b3f40fd95bff8c0cee82cd2e0c7f905a2d275d0b725"} Nov 04 05:47:18.871741 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.871647 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm" event={"ID":"8018ad9e-115b-47b3-9475-e6b06e8de89f","Type":"ContainerDied","Data":"64028dd1252cb6c434e3d14dfa079235e8f9332ac2db4d06bb3cce8134077dac"} Nov 04 05:47:18.871741 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.871687 4859 scope.go:122] "RemoveContainer" containerID="fd3328be4b32ed7e7db82b3f40fd95bff8c0cee82cd2e0c7f905a2d275d0b725" Nov 04 05:47:18.873421 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.873398 4859 generic.go:356] "Generic (PLEG): container finished" podID="7e5b156e-c723-400a-b76e-4d914ad58712" containerID="dc42e57ca337d7f0b9ca62ec7eb62f3114d35e3db6988407e20ac7db4f1e8760" exitCode=0 Nov 04 05:47:18.873520 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.873427 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" event={"ID":"7e5b156e-c723-400a-b76e-4d914ad58712","Type":"ContainerDied","Data":"dc42e57ca337d7f0b9ca62ec7eb62f3114d35e3db6988407e20ac7db4f1e8760"} Nov 04 05:47:18.873520 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.873443 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" Nov 04 05:47:18.873520 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.873476 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s" event={"ID":"7e5b156e-c723-400a-b76e-4d914ad58712","Type":"ContainerDied","Data":"e34411db6f73291ed53068ec6770a96ba2effb9046e0e8bb714c57b2c6703391"} Nov 04 05:47:18.952393 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.952369 4859 scope.go:122] "RemoveContainer" containerID="fd3328be4b32ed7e7db82b3f40fd95bff8c0cee82cd2e0c7f905a2d275d0b725" Nov 04 05:47:18.953021 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:47:18.952708 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd3328be4b32ed7e7db82b3f40fd95bff8c0cee82cd2e0c7f905a2d275d0b725\": container with ID starting with fd3328be4b32ed7e7db82b3f40fd95bff8c0cee82cd2e0c7f905a2d275d0b725 not found: ID does not exist" containerID="fd3328be4b32ed7e7db82b3f40fd95bff8c0cee82cd2e0c7f905a2d275d0b725" Nov 04 05:47:18.953021 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.952741 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd3328be4b32ed7e7db82b3f40fd95bff8c0cee82cd2e0c7f905a2d275d0b725"} err="failed to get container status \"fd3328be4b32ed7e7db82b3f40fd95bff8c0cee82cd2e0c7f905a2d275d0b725\": rpc error: code = NotFound desc = could not find container \"fd3328be4b32ed7e7db82b3f40fd95bff8c0cee82cd2e0c7f905a2d275d0b725\": container with ID starting with fd3328be4b32ed7e7db82b3f40fd95bff8c0cee82cd2e0c7f905a2d275d0b725 not found: ID does not exist" Nov 04 05:47:18.953021 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.952760 4859 scope.go:122] "RemoveContainer" containerID="dc42e57ca337d7f0b9ca62ec7eb62f3114d35e3db6988407e20ac7db4f1e8760" Nov 04 05:47:18.962225 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.962209 4859 scope.go:122] "RemoveContainer" containerID="dc42e57ca337d7f0b9ca62ec7eb62f3114d35e3db6988407e20ac7db4f1e8760" Nov 04 05:47:18.962523 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:47:18.962504 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc42e57ca337d7f0b9ca62ec7eb62f3114d35e3db6988407e20ac7db4f1e8760\": container with ID starting with dc42e57ca337d7f0b9ca62ec7eb62f3114d35e3db6988407e20ac7db4f1e8760 not found: ID does not exist" containerID="dc42e57ca337d7f0b9ca62ec7eb62f3114d35e3db6988407e20ac7db4f1e8760" Nov 04 05:47:18.962572 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:18.962531 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc42e57ca337d7f0b9ca62ec7eb62f3114d35e3db6988407e20ac7db4f1e8760"} err="failed to get container status \"dc42e57ca337d7f0b9ca62ec7eb62f3114d35e3db6988407e20ac7db4f1e8760\": rpc error: code = NotFound desc = could not find container \"dc42e57ca337d7f0b9ca62ec7eb62f3114d35e3db6988407e20ac7db4f1e8760\": container with ID starting with dc42e57ca337d7f0b9ca62ec7eb62f3114d35e3db6988407e20ac7db4f1e8760 not found: ID does not exist" Nov 04 05:47:19.122698 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.122652 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:19.123399 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.123380 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s"] Nov 04 05:47:19.126351 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.126326 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-fxz9s"] Nov 04 05:47:19.154455 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.154436 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:19.219436 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.219403 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-istio-envoy\") pod \"1987453d-4e27-42c8-9737-a6689b817176\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " Nov 04 05:47:19.219632 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.219449 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-workload-socket\") pod \"a6f3c605-6d79-4057-9203-c965c9084fbe\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " Nov 04 05:47:19.219632 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.219475 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-credential-socket\") pod \"1987453d-4e27-42c8-9737-a6689b817176\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " Nov 04 05:47:19.219632 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.219505 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-token\" (UniqueName: \"kubernetes.io/projected/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-token\") pod \"a6f3c605-6d79-4057-9203-c965c9084fbe\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " Nov 04 05:47:19.219632 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.219524 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-envoy\") pod \"a6f3c605-6d79-4057-9203-c965c9084fbe\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " Nov 04 05:47:19.219632 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.219548 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-podinfo\") pod \"a6f3c605-6d79-4057-9203-c965c9084fbe\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " Nov 04 05:47:19.219632 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.219599 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-workload-certs\") pod \"a6f3c605-6d79-4057-9203-c965c9084fbe\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " Nov 04 05:47:19.219984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.219648 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1987453d-4e27-42c8-9737-a6689b817176-istio-token\" (UniqueName: \"kubernetes.io/projected/1987453d-4e27-42c8-9737-a6689b817176-istio-token\") pod \"1987453d-4e27-42c8-9737-a6689b817176\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " Nov 04 05:47:19.219984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.219724 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-workload-certs\") pod \"1987453d-4e27-42c8-9737-a6689b817176\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " Nov 04 05:47:19.219984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.219753 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a6f3c605-6d79-4057-9203-c965c9084fbe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6f3c605-6d79-4057-9203-c965c9084fbe-istiod-ca-cert\") pod \"a6f3c605-6d79-4057-9203-c965c9084fbe\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " Nov 04 05:47:19.219984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.219778 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-istio-data\") pod \"1987453d-4e27-42c8-9737-a6689b817176\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " Nov 04 05:47:19.219984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.219823 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-workload-socket" pod "a6f3c605-6d79-4057-9203-c965c9084fbe" (UID: "a6f3c605-6d79-4057-9203-c965c9084fbe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.219984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.219836 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-workload-socket\") pod \"1987453d-4e27-42c8-9737-a6689b817176\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " Nov 04 05:47:19.219984 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.219937 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-data\") pod \"a6f3c605-6d79-4057-9203-c965c9084fbe\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " Nov 04 05:47:19.220346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.219991 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-credential-socket\") pod \"a6f3c605-6d79-4057-9203-c965c9084fbe\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " Nov 04 05:47:19.220346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.220042 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1987453d-4e27-42c8-9737-a6689b817176-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1987453d-4e27-42c8-9737-a6689b817176-istio-podinfo\") pod \"1987453d-4e27-42c8-9737-a6689b817176\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " Nov 04 05:47:19.220346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.220076 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1987453d-4e27-42c8-9737-a6689b817176-kube-api-access-8k89j\" (UniqueName: \"kubernetes.io/projected/1987453d-4e27-42c8-9737-a6689b817176-kube-api-access-8k89j\") pod \"1987453d-4e27-42c8-9737-a6689b817176\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " Nov 04 05:47:19.220346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.220172 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1987453d-4e27-42c8-9737-a6689b817176-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1987453d-4e27-42c8-9737-a6689b817176-istiod-ca-cert\") pod \"1987453d-4e27-42c8-9737-a6689b817176\" (UID: \"1987453d-4e27-42c8-9737-a6689b817176\") " Nov 04 05:47:19.220346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.220205 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-workload-certs" pod "a6f3c605-6d79-4057-9203-c965c9084fbe" (UID: "a6f3c605-6d79-4057-9203-c965c9084fbe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.220346 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.220229 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a6f3c605-6d79-4057-9203-c965c9084fbe-kube-api-access-snpmx\" (UniqueName: \"kubernetes.io/projected/a6f3c605-6d79-4057-9203-c965c9084fbe-kube-api-access-snpmx\") pod \"a6f3c605-6d79-4057-9203-c965c9084fbe\" (UID: \"a6f3c605-6d79-4057-9203-c965c9084fbe\") " Nov 04 05:47:19.221499 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.220726 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-credential-socket" pod "1987453d-4e27-42c8-9737-a6689b817176" (UID: "1987453d-4e27-42c8-9737-a6689b817176"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.221499 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.221098 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-workload-certs" pod "1987453d-4e27-42c8-9737-a6689b817176" (UID: "1987453d-4e27-42c8-9737-a6689b817176"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.222188 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.221536 4859 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-workload-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.222188 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.221564 4859 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-credential-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.222188 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.221594 4859 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-workload-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.222188 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.221610 4859 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-workload-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.222188 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.221865 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6f3c605-6d79-4057-9203-c965c9084fbe-istiod-ca-cert" pod "a6f3c605-6d79-4057-9203-c965c9084fbe" (UID: "a6f3c605-6d79-4057-9203-c965c9084fbe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:19.222188 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.221971 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-istio-data" pod "1987453d-4e27-42c8-9737-a6689b817176" (UID: "1987453d-4e27-42c8-9737-a6689b817176"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.222188 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.221976 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-data" pod "a6f3c605-6d79-4057-9203-c965c9084fbe" (UID: "a6f3c605-6d79-4057-9203-c965c9084fbe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.222188 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.222193 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-credential-socket" pod "a6f3c605-6d79-4057-9203-c965c9084fbe" (UID: "a6f3c605-6d79-4057-9203-c965c9084fbe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.222651 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.222236 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-token" pod "a6f3c605-6d79-4057-9203-c965c9084fbe" (UID: "a6f3c605-6d79-4057-9203-c965c9084fbe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:19.222651 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.222309 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1987453d-4e27-42c8-9737-a6689b817176-istiod-ca-cert" pod "1987453d-4e27-42c8-9737-a6689b817176" (UID: "1987453d-4e27-42c8-9737-a6689b817176"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:19.222651 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.222574 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-podinfo" pod "a6f3c605-6d79-4057-9203-c965c9084fbe" (UID: "a6f3c605-6d79-4057-9203-c965c9084fbe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:19.222651 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.222641 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-istio-envoy" pod "1987453d-4e27-42c8-9737-a6689b817176" (UID: "1987453d-4e27-42c8-9737-a6689b817176"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.222882 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.222718 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-workload-socket" pod "1987453d-4e27-42c8-9737-a6689b817176" (UID: "1987453d-4e27-42c8-9737-a6689b817176"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.223396 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.223366 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-envoy" pod "a6f3c605-6d79-4057-9203-c965c9084fbe" (UID: "a6f3c605-6d79-4057-9203-c965c9084fbe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.224065 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.224043 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6f3c605-6d79-4057-9203-c965c9084fbe-kube-api-access-snpmx" pod "a6f3c605-6d79-4057-9203-c965c9084fbe" (UID: "a6f3c605-6d79-4057-9203-c965c9084fbe"). InnerVolumeSpecName "kube-api-access-snpmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:19.224490 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.224463 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1987453d-4e27-42c8-9737-a6689b817176-istio-token" pod "1987453d-4e27-42c8-9737-a6689b817176" (UID: "1987453d-4e27-42c8-9737-a6689b817176"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:19.224556 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.224540 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1987453d-4e27-42c8-9737-a6689b817176-kube-api-access-8k89j" pod "1987453d-4e27-42c8-9737-a6689b817176" (UID: "1987453d-4e27-42c8-9737-a6689b817176"). InnerVolumeSpecName "kube-api-access-8k89j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:19.224980 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.224956 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1987453d-4e27-42c8-9737-a6689b817176-istio-podinfo" pod "1987453d-4e27-42c8-9737-a6689b817176" (UID: "1987453d-4e27-42c8-9737-a6689b817176"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:19.244770 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.244746 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm"] Nov 04 05:47:19.249915 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.249894 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-vxbhm"] Nov 04 05:47:19.323002 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.322969 4859 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6f3c605-6d79-4057-9203-c965c9084fbe-istiod-ca-cert\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.323002 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.322995 4859 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-istio-data\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.323002 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.323005 4859 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-workload-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.323223 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.323015 4859 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-data\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.323223 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.323024 4859 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-credential-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.323223 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.323033 4859 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1987453d-4e27-42c8-9737-a6689b817176-istio-podinfo\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.323223 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.323045 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8k89j\" (UniqueName: \"kubernetes.io/projected/1987453d-4e27-42c8-9737-a6689b817176-kube-api-access-8k89j\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.323223 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.323054 4859 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1987453d-4e27-42c8-9737-a6689b817176-istiod-ca-cert\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.323223 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.323062 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snpmx\" (UniqueName: \"kubernetes.io/projected/a6f3c605-6d79-4057-9203-c965c9084fbe-kube-api-access-snpmx\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.323223 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.323072 4859 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1987453d-4e27-42c8-9737-a6689b817176-istio-envoy\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.323223 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.323080 4859 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.323223 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.323088 4859 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-envoy\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.323223 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.323096 4859 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6f3c605-6d79-4057-9203-c965c9084fbe-istio-podinfo\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.323223 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.323104 4859 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1987453d-4e27-42c8-9737-a6689b817176-istio-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.631145 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.631106 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Nov 04 05:47:19.631590 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.631151 4859 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" podUID="3d2b29d5-afab-4bfd-878f-615e74d45cfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Nov 04 05:47:19.735993 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.735958 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj"] Nov 04 05:47:19.736890 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.736858 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" podUID="6b3635c9-2a57-4ebb-a371-877a21946f46" containerName="istio-proxy" containerID="cri-o://05d563eba0a844b227d0c111f4e1368256207a7b7de9934fe437f5be193d4889" gracePeriod=30 Nov 04 05:47:19.740044 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.740019 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd"] Nov 04 05:47:19.740318 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.740278 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" podUID="2d4d34a1-0909-4690-8b21-e90d76afdfde" containerName="istio-proxy" containerID="cri-o://ef7b3c953a7643bc1b84e8cca637100d151954ed127bee8c0f49a885d8f2d08b" gracePeriod=30 Nov 04 05:47:19.746337 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.746314 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7"] Nov 04 05:47:19.746488 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.746468 4859 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" podUID="3d2b29d5-afab-4bfd-878f-615e74d45cfa" containerName="istio-proxy" containerID="cri-o://ceb6029c8211f6343ba8bb39490265c9df09039e47f06b20afef36bec61058e5" gracePeriod=30 Nov 04 05:47:19.882430 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.882345 4859 generic.go:356] "Generic (PLEG): container finished" podID="a6f3c605-6d79-4057-9203-c965c9084fbe" containerID="526e4b1fdf27556791068b48accc678deeca98be4e7415346073541aacdea80d" exitCode=0 Nov 04 05:47:19.882430 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.882403 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" Nov 04 05:47:19.882702 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.882428 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" event={"ID":"a6f3c605-6d79-4057-9203-c965c9084fbe","Type":"ContainerDied","Data":"526e4b1fdf27556791068b48accc678deeca98be4e7415346073541aacdea80d"} Nov 04 05:47:19.882702 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.882481 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484" event={"ID":"a6f3c605-6d79-4057-9203-c965c9084fbe","Type":"ContainerDied","Data":"219eedeef30059b29c6328042602e22c608c344455d97a0148c22d5a04ad0a01"} Nov 04 05:47:19.882702 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.882505 4859 scope.go:122] "RemoveContainer" containerID="526e4b1fdf27556791068b48accc678deeca98be4e7415346073541aacdea80d" Nov 04 05:47:19.884999 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.884969 4859 generic.go:356] "Generic (PLEG): container finished" podID="1987453d-4e27-42c8-9737-a6689b817176" containerID="787b80e46ead8afccf48b86abb1b8987d188a43d08a2cc02da0806d8497a2277" exitCode=0 Nov 04 05:47:19.885108 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.885022 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" Nov 04 05:47:19.885108 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.885057 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" event={"ID":"1987453d-4e27-42c8-9737-a6689b817176","Type":"ContainerDied","Data":"787b80e46ead8afccf48b86abb1b8987d188a43d08a2cc02da0806d8497a2277"} Nov 04 05:47:19.885108 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.885096 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9" event={"ID":"1987453d-4e27-42c8-9737-a6689b817176","Type":"ContainerDied","Data":"b1c3ba49f7db1121ae56ac6976fc21edc45dd6ce75535db32831bd34ec0a3d87"} Nov 04 05:47:19.893571 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.893555 4859 scope.go:122] "RemoveContainer" containerID="526e4b1fdf27556791068b48accc678deeca98be4e7415346073541aacdea80d" Nov 04 05:47:19.893927 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:47:19.893909 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"526e4b1fdf27556791068b48accc678deeca98be4e7415346073541aacdea80d\": container with ID starting with 526e4b1fdf27556791068b48accc678deeca98be4e7415346073541aacdea80d not found: ID does not exist" containerID="526e4b1fdf27556791068b48accc678deeca98be4e7415346073541aacdea80d" Nov 04 05:47:19.893986 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.893934 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"526e4b1fdf27556791068b48accc678deeca98be4e7415346073541aacdea80d"} err="failed to get container status \"526e4b1fdf27556791068b48accc678deeca98be4e7415346073541aacdea80d\": rpc error: code = NotFound desc = could not find container \"526e4b1fdf27556791068b48accc678deeca98be4e7415346073541aacdea80d\": container with ID starting with 526e4b1fdf27556791068b48accc678deeca98be4e7415346073541aacdea80d not found: ID does not exist" Nov 04 05:47:19.893986 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.893950 4859 scope.go:122] "RemoveContainer" containerID="787b80e46ead8afccf48b86abb1b8987d188a43d08a2cc02da0806d8497a2277" Nov 04 05:47:19.902282 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.902268 4859 scope.go:122] "RemoveContainer" containerID="787b80e46ead8afccf48b86abb1b8987d188a43d08a2cc02da0806d8497a2277" Nov 04 05:47:19.902575 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:47:19.902554 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"787b80e46ead8afccf48b86abb1b8987d188a43d08a2cc02da0806d8497a2277\": container with ID starting with 787b80e46ead8afccf48b86abb1b8987d188a43d08a2cc02da0806d8497a2277 not found: ID does not exist" containerID="787b80e46ead8afccf48b86abb1b8987d188a43d08a2cc02da0806d8497a2277" Nov 04 05:47:19.902673 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:19.902580 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"787b80e46ead8afccf48b86abb1b8987d188a43d08a2cc02da0806d8497a2277"} err="failed to get container status \"787b80e46ead8afccf48b86abb1b8987d188a43d08a2cc02da0806d8497a2277\": rpc error: code = NotFound desc = could not find container \"787b80e46ead8afccf48b86abb1b8987d188a43d08a2cc02da0806d8497a2277\": container with ID starting with 787b80e46ead8afccf48b86abb1b8987d188a43d08a2cc02da0806d8497a2277 not found: ID does not exist" Nov 04 05:47:20.116571 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:20.116501 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e5b156e-c723-400a-b76e-4d914ad58712" path="/var/lib/kubelet/pods/7e5b156e-c723-400a-b76e-4d914ad58712/volumes" Nov 04 05:47:20.116960 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:20.116946 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8018ad9e-115b-47b3-9475-e6b06e8de89f" path="/var/lib/kubelet/pods/8018ad9e-115b-47b3-9475-e6b06e8de89f/volumes" Nov 04 05:47:20.224355 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:20.224314 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:47:20.224500 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:20.224399 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" podUID="2d4d34a1-0909-4690-8b21-e90d76afdfde" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:47:20.253166 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:20.253131 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484"] Nov 04 05:47:20.256907 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:20.256881 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-2k484"] Nov 04 05:47:20.273757 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:20.273724 4859 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:47:20.273851 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:20.273790 4859 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" podUID="6b3635c9-2a57-4ebb-a371-877a21946f46" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:47:20.373072 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:20.373046 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9"] Nov 04 05:47:20.377189 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:20.377165 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-9hvq9"] Nov 04 05:47:22.115944 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:22.115911 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1987453d-4e27-42c8-9737-a6689b817176" path="/var/lib/kubelet/pods/1987453d-4e27-42c8-9737-a6689b817176/volumes" Nov 04 05:47:22.116547 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:22.116526 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a6f3c605-6d79-4057-9203-c965c9084fbe" path="/var/lib/kubelet/pods/a6f3c605-6d79-4057-9203-c965c9084fbe/volumes" Nov 04 05:47:24.918244 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:24.918207 4859 generic.go:356] "Generic (PLEG): container finished" podID="6b3635c9-2a57-4ebb-a371-877a21946f46" containerID="05d563eba0a844b227d0c111f4e1368256207a7b7de9934fe437f5be193d4889" exitCode=0 Nov 04 05:47:24.918685 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:24.918402 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" event={"ID":"6b3635c9-2a57-4ebb-a371-877a21946f46","Type":"ContainerDied","Data":"05d563eba0a844b227d0c111f4e1368256207a7b7de9934fe437f5be193d4889"} Nov 04 05:47:24.920574 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:24.920540 4859 generic.go:356] "Generic (PLEG): container finished" podID="2d4d34a1-0909-4690-8b21-e90d76afdfde" containerID="ef7b3c953a7643bc1b84e8cca637100d151954ed127bee8c0f49a885d8f2d08b" exitCode=0 Nov 04 05:47:24.920574 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:24.920572 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" event={"ID":"2d4d34a1-0909-4690-8b21-e90d76afdfde","Type":"ContainerDied","Data":"ef7b3c953a7643bc1b84e8cca637100d151954ed127bee8c0f49a885d8f2d08b"} Nov 04 05:47:25.004123 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.004100 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:47:25.026059 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.026038 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:47:25.084026 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084001 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-envoy\") pod \"2d4d34a1-0909-4690-8b21-e90d76afdfde\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " Nov 04 05:47:25.084147 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084035 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-workload-socket\") pod \"2d4d34a1-0909-4690-8b21-e90d76afdfde\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " Nov 04 05:47:25.084147 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084062 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2d4d34a1-0909-4690-8b21-e90d76afdfde-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d4d34a1-0909-4690-8b21-e90d76afdfde-istiod-ca-cert\") pod \"2d4d34a1-0909-4690-8b21-e90d76afdfde\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " Nov 04 05:47:25.084147 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084121 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-workload-certs\") pod \"2d4d34a1-0909-4690-8b21-e90d76afdfde\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " Nov 04 05:47:25.084335 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084153 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-podinfo\") pod \"2d4d34a1-0909-4690-8b21-e90d76afdfde\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " Nov 04 05:47:25.084335 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084184 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-data\") pod \"2d4d34a1-0909-4690-8b21-e90d76afdfde\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " Nov 04 05:47:25.084335 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084236 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d4d34a1-0909-4690-8b21-e90d76afdfde-kube-api-access-qjm75\" (UniqueName: \"kubernetes.io/projected/2d4d34a1-0909-4690-8b21-e90d76afdfde-kube-api-access-qjm75\") pod \"2d4d34a1-0909-4690-8b21-e90d76afdfde\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " Nov 04 05:47:25.084335 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084325 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-credential-socket\") pod \"2d4d34a1-0909-4690-8b21-e90d76afdfde\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " Nov 04 05:47:25.084541 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084379 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-token\" (UniqueName: \"kubernetes.io/projected/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-token\") pod \"2d4d34a1-0909-4690-8b21-e90d76afdfde\" (UID: \"2d4d34a1-0909-4690-8b21-e90d76afdfde\") " Nov 04 05:47:25.084541 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084379 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-workload-socket" pod "2d4d34a1-0909-4690-8b21-e90d76afdfde" (UID: "2d4d34a1-0909-4690-8b21-e90d76afdfde"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.084541 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084428 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-workload-certs" pod "2d4d34a1-0909-4690-8b21-e90d76afdfde" (UID: "2d4d34a1-0909-4690-8b21-e90d76afdfde"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.084730 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084541 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d4d34a1-0909-4690-8b21-e90d76afdfde-istiod-ca-cert" pod "2d4d34a1-0909-4690-8b21-e90d76afdfde" (UID: "2d4d34a1-0909-4690-8b21-e90d76afdfde"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:25.084730 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084620 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-data" pod "2d4d34a1-0909-4690-8b21-e90d76afdfde" (UID: "2d4d34a1-0909-4690-8b21-e90d76afdfde"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.084730 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084638 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-credential-socket" pod "2d4d34a1-0909-4690-8b21-e90d76afdfde" (UID: "2d4d34a1-0909-4690-8b21-e90d76afdfde"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.084882 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084813 4859 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-credential-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.084882 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084834 4859 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-workload-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.084882 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084844 4859 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d4d34a1-0909-4690-8b21-e90d76afdfde-istiod-ca-cert\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.084882 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084854 4859 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-workload-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.084882 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.084862 4859 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-data\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.086285 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.086261 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-envoy" pod "2d4d34a1-0909-4690-8b21-e90d76afdfde" (UID: "2d4d34a1-0909-4690-8b21-e90d76afdfde"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.086385 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.086366 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-token" pod "2d4d34a1-0909-4690-8b21-e90d76afdfde" (UID: "2d4d34a1-0909-4690-8b21-e90d76afdfde"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:25.086441 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.086402 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d4d34a1-0909-4690-8b21-e90d76afdfde-kube-api-access-qjm75" pod "2d4d34a1-0909-4690-8b21-e90d76afdfde" (UID: "2d4d34a1-0909-4690-8b21-e90d76afdfde"). InnerVolumeSpecName "kube-api-access-qjm75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:25.086478 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.086446 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-podinfo" pod "2d4d34a1-0909-4690-8b21-e90d76afdfde" (UID: "2d4d34a1-0909-4690-8b21-e90d76afdfde"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:25.185650 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.185622 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b3635c9-2a57-4ebb-a371-877a21946f46-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3635c9-2a57-4ebb-a371-877a21946f46-istiod-ca-cert\") pod \"6b3635c9-2a57-4ebb-a371-877a21946f46\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " Nov 04 05:47:25.185801 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.185695 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-podinfo\") pod \"6b3635c9-2a57-4ebb-a371-877a21946f46\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " Nov 04 05:47:25.185801 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.185789 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-credential-socket\") pod \"6b3635c9-2a57-4ebb-a371-877a21946f46\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " Nov 04 05:47:25.185906 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.185832 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-data\") pod \"6b3635c9-2a57-4ebb-a371-877a21946f46\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " Nov 04 05:47:25.185906 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.185880 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-workload-socket\") pod \"6b3635c9-2a57-4ebb-a371-877a21946f46\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " Nov 04 05:47:25.186010 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.185937 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b3635c9-2a57-4ebb-a371-877a21946f46-kube-api-access-mhxfv\" (UniqueName: \"kubernetes.io/projected/6b3635c9-2a57-4ebb-a371-877a21946f46-kube-api-access-mhxfv\") pod \"6b3635c9-2a57-4ebb-a371-877a21946f46\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " Nov 04 05:47:25.186010 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.185981 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-workload-certs\") pod \"6b3635c9-2a57-4ebb-a371-877a21946f46\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " Nov 04 05:47:25.186111 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.186020 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-token\" (UniqueName: \"kubernetes.io/projected/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-token\") pod \"6b3635c9-2a57-4ebb-a371-877a21946f46\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " Nov 04 05:47:25.186111 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.186088 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-envoy\") pod \"6b3635c9-2a57-4ebb-a371-877a21946f46\" (UID: \"6b3635c9-2a57-4ebb-a371-877a21946f46\") " Nov 04 05:47:25.186215 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.186115 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-data" pod "6b3635c9-2a57-4ebb-a371-877a21946f46" (UID: "6b3635c9-2a57-4ebb-a371-877a21946f46"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.186215 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.186131 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-credential-socket" pod "6b3635c9-2a57-4ebb-a371-877a21946f46" (UID: "6b3635c9-2a57-4ebb-a371-877a21946f46"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.186215 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.186177 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b3635c9-2a57-4ebb-a371-877a21946f46-istiod-ca-cert" pod "6b3635c9-2a57-4ebb-a371-877a21946f46" (UID: "6b3635c9-2a57-4ebb-a371-877a21946f46"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:25.186902 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.186429 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-workload-certs" pod "6b3635c9-2a57-4ebb-a371-877a21946f46" (UID: "6b3635c9-2a57-4ebb-a371-877a21946f46"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.186902 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.186527 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjm75\" (UniqueName: \"kubernetes.io/projected/2d4d34a1-0909-4690-8b21-e90d76afdfde-kube-api-access-qjm75\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.186902 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.186548 4859 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-workload-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.186902 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.186563 4859 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.186902 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.186576 4859 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-envoy\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.186902 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.186587 4859 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3635c9-2a57-4ebb-a371-877a21946f46-istiod-ca-cert\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.186902 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.186599 4859 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-credential-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.186902 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.186613 4859 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-data\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.186902 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.186624 4859 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d4d34a1-0909-4690-8b21-e90d76afdfde-istio-podinfo\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.186902 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.186803 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-workload-socket" pod "6b3635c9-2a57-4ebb-a371-877a21946f46" (UID: "6b3635c9-2a57-4ebb-a371-877a21946f46"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.188917 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.188880 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-podinfo" pod "6b3635c9-2a57-4ebb-a371-877a21946f46" (UID: "6b3635c9-2a57-4ebb-a371-877a21946f46"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:25.189256 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.189222 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b3635c9-2a57-4ebb-a371-877a21946f46-kube-api-access-mhxfv" pod "6b3635c9-2a57-4ebb-a371-877a21946f46" (UID: "6b3635c9-2a57-4ebb-a371-877a21946f46"). InnerVolumeSpecName "kube-api-access-mhxfv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:25.189439 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.189419 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-token" pod "6b3635c9-2a57-4ebb-a371-877a21946f46" (UID: "6b3635c9-2a57-4ebb-a371-877a21946f46"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:25.190013 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.189991 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-envoy" pod "6b3635c9-2a57-4ebb-a371-877a21946f46" (UID: "6b3635c9-2a57-4ebb-a371-877a21946f46"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.287329 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.287265 4859 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-podinfo\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.287329 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.287289 4859 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-workload-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.287329 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.287300 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhxfv\" (UniqueName: \"kubernetes.io/projected/6b3635c9-2a57-4ebb-a371-877a21946f46-kube-api-access-mhxfv\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.287329 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.287311 4859 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.287329 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.287322 4859 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3635c9-2a57-4ebb-a371-877a21946f46-istio-envoy\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.900593 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.900570 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:25.926215 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.926190 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" Nov 04 05:47:25.926566 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.926213 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj" event={"ID":"6b3635c9-2a57-4ebb-a371-877a21946f46","Type":"ContainerDied","Data":"b6435b6fadcff9c5557178133f29c4b40b8feac586489f5e5b005a1a9ace2731"} Nov 04 05:47:25.926566 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.926273 4859 scope.go:122] "RemoveContainer" containerID="05d563eba0a844b227d0c111f4e1368256207a7b7de9934fe437f5be193d4889" Nov 04 05:47:25.927614 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.927583 4859 generic.go:356] "Generic (PLEG): container finished" podID="3d2b29d5-afab-4bfd-878f-615e74d45cfa" containerID="ceb6029c8211f6343ba8bb39490265c9df09039e47f06b20afef36bec61058e5" exitCode=0 Nov 04 05:47:25.927748 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.927625 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" Nov 04 05:47:25.927748 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.927633 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" event={"ID":"3d2b29d5-afab-4bfd-878f-615e74d45cfa","Type":"ContainerDied","Data":"ceb6029c8211f6343ba8bb39490265c9df09039e47f06b20afef36bec61058e5"} Nov 04 05:47:25.927748 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.927703 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7" event={"ID":"3d2b29d5-afab-4bfd-878f-615e74d45cfa","Type":"ContainerDied","Data":"a6e01a11edec078ceaf41a83f9545e7b66da86b4a3a192b8364417706c2fd795"} Nov 04 05:47:25.929156 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.929130 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" event={"ID":"2d4d34a1-0909-4690-8b21-e90d76afdfde","Type":"ContainerDied","Data":"3bcbc515e8ff6609d8f32d8944fe52f0f68737353e932938295d3c1d371c255c"} Nov 04 05:47:25.929257 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.929159 4859 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd" Nov 04 05:47:25.937853 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.937839 4859 scope.go:122] "RemoveContainer" containerID="ceb6029c8211f6343ba8bb39490265c9df09039e47f06b20afef36bec61058e5" Nov 04 05:47:25.946155 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.946136 4859 scope.go:122] "RemoveContainer" containerID="ceb6029c8211f6343ba8bb39490265c9df09039e47f06b20afef36bec61058e5" Nov 04 05:47:25.946411 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:47:25.946394 4859 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceb6029c8211f6343ba8bb39490265c9df09039e47f06b20afef36bec61058e5\": container with ID starting with ceb6029c8211f6343ba8bb39490265c9df09039e47f06b20afef36bec61058e5 not found: ID does not exist" containerID="ceb6029c8211f6343ba8bb39490265c9df09039e47f06b20afef36bec61058e5" Nov 04 05:47:25.946458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.946423 4859 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceb6029c8211f6343ba8bb39490265c9df09039e47f06b20afef36bec61058e5"} err="failed to get container status \"ceb6029c8211f6343ba8bb39490265c9df09039e47f06b20afef36bec61058e5\": rpc error: code = NotFound desc = could not find container \"ceb6029c8211f6343ba8bb39490265c9df09039e47f06b20afef36bec61058e5\": container with ID starting with ceb6029c8211f6343ba8bb39490265c9df09039e47f06b20afef36bec61058e5 not found: ID does not exist" Nov 04 05:47:25.946458 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.946439 4859 scope.go:122] "RemoveContainer" containerID="ef7b3c953a7643bc1b84e8cca637100d151954ed127bee8c0f49a885d8f2d08b" Nov 04 05:47:25.994337 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.994272 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-data\") pod \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " Nov 04 05:47:25.994337 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.994326 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-workload-socket\") pod \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " Nov 04 05:47:25.994476 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.994355 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-envoy\") pod \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " Nov 04 05:47:25.994476 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.994410 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-workload-certs\") pod \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " Nov 04 05:47:25.994476 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.994441 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d2b29d5-afab-4bfd-878f-615e74d45cfa-kube-api-access-2qblw\" (UniqueName: \"kubernetes.io/projected/3d2b29d5-afab-4bfd-878f-615e74d45cfa-kube-api-access-2qblw\") pod \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " Nov 04 05:47:25.994625 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.994501 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-podinfo\") pod \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " Nov 04 05:47:25.994625 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.994540 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-token\" (UniqueName: \"kubernetes.io/projected/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-token\") pod \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " Nov 04 05:47:25.994625 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.994563 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-data" pod "3d2b29d5-afab-4bfd-878f-615e74d45cfa" (UID: "3d2b29d5-afab-4bfd-878f-615e74d45cfa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.994625 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.994575 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-workload-socket" pod "3d2b29d5-afab-4bfd-878f-615e74d45cfa" (UID: "3d2b29d5-afab-4bfd-878f-615e74d45cfa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.994625 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.994591 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istiod-ca-cert\") pod \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " Nov 04 05:47:25.994882 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.994732 4859 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-credential-socket\") pod \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\" (UID: \"3d2b29d5-afab-4bfd-878f-615e74d45cfa\") " Nov 04 05:47:25.994882 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.994792 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-workload-certs" pod "3d2b29d5-afab-4bfd-878f-615e74d45cfa" (UID: "3d2b29d5-afab-4bfd-878f-615e74d45cfa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.995053 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.995034 4859 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-workload-certs\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.995142 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.995059 4859 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-data\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.995142 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.995076 4859 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-workload-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.995325 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.995204 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istiod-ca-cert" pod "3d2b29d5-afab-4bfd-878f-615e74d45cfa" (UID: "3d2b29d5-afab-4bfd-878f-615e74d45cfa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:25.995325 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.995274 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-credential-socket" pod "3d2b29d5-afab-4bfd-878f-615e74d45cfa" (UID: "3d2b29d5-afab-4bfd-878f-615e74d45cfa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.996848 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.996813 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d2b29d5-afab-4bfd-878f-615e74d45cfa-kube-api-access-2qblw" pod "3d2b29d5-afab-4bfd-878f-615e74d45cfa" (UID: "3d2b29d5-afab-4bfd-878f-615e74d45cfa"). InnerVolumeSpecName "kube-api-access-2qblw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:25.997214 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.997179 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-token" pod "3d2b29d5-afab-4bfd-878f-615e74d45cfa" (UID: "3d2b29d5-afab-4bfd-878f-615e74d45cfa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:25.997381 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.997359 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-envoy" pod "3d2b29d5-afab-4bfd-878f-615e74d45cfa" (UID: "3d2b29d5-afab-4bfd-878f-615e74d45cfa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.997501 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:25.997480 4859 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-podinfo" pod "3d2b29d5-afab-4bfd-878f-615e74d45cfa" (UID: "3d2b29d5-afab-4bfd-878f-615e74d45cfa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:26.096357 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:26.096325 4859 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-credential-socket\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.096357 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:26.096351 4859 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-envoy\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.096357 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:26.096362 4859 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qblw\" (UniqueName: \"kubernetes.io/projected/3d2b29d5-afab-4bfd-878f-615e74d45cfa-kube-api-access-2qblw\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.096651 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:26.096371 4859 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-podinfo\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.096651 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:26.096381 4859 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istio-token\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.096651 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:26.096389 4859 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d2b29d5-afab-4bfd-878f-615e74d45cfa-istiod-ca-cert\") on node \"ip-10-0-133-143.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.206397 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:26.206370 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd"] Nov 04 05:47:26.210781 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:26.210760 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-sswmd"] Nov 04 05:47:26.332377 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:26.332303 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj"] Nov 04 05:47:26.336070 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:26.336044 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-9nmqj"] Nov 04 05:47:26.513494 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:26.513465 4859 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7"] Nov 04 05:47:26.517713 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:26.517689 4859 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-t4cr7"] Nov 04 05:47:28.115894 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:28.115860 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d4d34a1-0909-4690-8b21-e90d76afdfde" path="/var/lib/kubelet/pods/2d4d34a1-0909-4690-8b21-e90d76afdfde/volumes" Nov 04 05:47:28.117011 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:28.116973 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d2b29d5-afab-4bfd-878f-615e74d45cfa" path="/var/lib/kubelet/pods/3d2b29d5-afab-4bfd-878f-615e74d45cfa/volumes" Nov 04 05:47:28.117600 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:28.117545 4859 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b3635c9-2a57-4ebb-a371-877a21946f46" path="/var/lib/kubelet/pods/6b3635c9-2a57-4ebb-a371-877a21946f46/volumes" Nov 04 05:47:28.340742 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:47:28.340702 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:47:34.805576 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:34.805543 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-m8srk_2a69b293-4aa6-4f66-b471-477cf841ce67/manager/0.log" Nov 04 05:47:35.887652 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:35.887626 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-w5msn_8972ea5d-5467-4935-b341-d3fbd7750a21/server/0.log" Nov 04 05:47:36.129017 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:36.128993 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-rv57b_0cf6a57d-3d4b-4aef-913f-c2fd31147260/manager/0.log" Nov 04 05:47:36.589146 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:36.589120 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-cjkv4_1b132e4f-1cda-4c76-820a-b81feb73ad40/proxy/0.log" Nov 04 05:47:37.629534 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:37.629510 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5cf85ff9fd-9mt9t_a691dddb-3a39-4bc1-8add-c5b2668c869d/maas-api/0.log" Nov 04 05:47:47.190562 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:47.190533 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-z4xcr_e371ee7d-fd98-4f76-b0ca-01dc7f636ba3/discovery/0.log" Nov 04 05:47:47.589578 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:47.589489 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c6f4696b9-jdr4t_ccec3278-ee58-40cc-a076-6380de7e1f72/kube-auth-proxy/0.log" Nov 04 05:47:48.579122 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:48.579087 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-t77lc_be1382e4-d585-4e8a-b5e5-f072ea92f29a/istio-proxy/0.log" Nov 04 05:47:49.325305 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:49.325278 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-chjgw_78647993-f015-4d14-a8df-d6f640bc9508/payload-pre-processing/0.log" Nov 04 05:47:49.790408 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:49.790380 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58c967f969-4xztc_c87f7dad-1181-4db5-b846-05d8fb0a33d5/router/0.log" Nov 04 05:47:51.130851 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:51.130818 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w_78899087-8b82-4eef-a7b1-70e2adb9665d/main/0.log" Nov 04 05:47:51.421101 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:51.421071 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-6w29d_4fdb4e8c-8202-4eb9-80a9-06c460502eb2/main/0.log" Nov 04 05:47:51.708250 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:51.708181 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-k6jv8_e15551cb-d922-4daf-b031-bfe47a443e54/main/0.log" Nov 04 05:47:51.939363 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:51.939337 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-t6p2k_8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3/main/0.log" Nov 04 05:47:52.407396 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:47:52.407369 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-ggvl5_e4282bcb-4054-4202-8513-1392d158ae81/main/0.log" Nov 04 05:47:58.341588 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:47:58.341547 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:48:04.160646 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:48:04.160616 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-m8srk_2a69b293-4aa6-4f66-b471-477cf841ce67/manager/0.log" Nov 04 05:48:05.329107 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:48:05.329079 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-w5msn_8972ea5d-5467-4935-b341-d3fbd7750a21/server/0.log" Nov 04 05:48:05.585531 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:48:05.585449 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-rv57b_0cf6a57d-3d4b-4aef-913f-c2fd31147260/manager/0.log" Nov 04 05:48:06.038903 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:48:06.038868 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-cjkv4_1b132e4f-1cda-4c76-820a-b81feb73ad40/proxy/0.log" Nov 04 05:48:06.638098 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:48:06.638073 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5cf85ff9fd-9mt9t_a691dddb-3a39-4bc1-8add-c5b2668c869d/maas-api/0.log" Nov 04 05:48:14.670030 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:48:14.669996 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-z4xcr_e371ee7d-fd98-4f76-b0ca-01dc7f636ba3/discovery/0.log" Nov 04 05:48:15.086466 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:48:15.086380 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c6f4696b9-jdr4t_ccec3278-ee58-40cc-a076-6380de7e1f72/kube-auth-proxy/0.log" Nov 04 05:48:16.145812 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:48:16.145780 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-t77lc_be1382e4-d585-4e8a-b5e5-f072ea92f29a/istio-proxy/0.log" Nov 04 05:48:17.046223 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:48:17.046185 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-chjgw_78647993-f015-4d14-a8df-d6f640bc9508/payload-pre-processing/0.log" Nov 04 05:48:17.550841 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:48:17.550809 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58c967f969-4xztc_c87f7dad-1181-4db5-b846-05d8fb0a33d5/router/0.log" Nov 04 05:48:18.221402 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:48:18.221375 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-9xl7w_78899087-8b82-4eef-a7b1-70e2adb9665d/main/0.log" Nov 04 05:48:18.525428 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:48:18.525350 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-6w29d_4fdb4e8c-8202-4eb9-80a9-06c460502eb2/main/0.log" Nov 04 05:48:18.781620 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:48:18.781543 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-k6jv8_e15551cb-d922-4daf-b031-bfe47a443e54/main/0.log" Nov 04 05:48:19.048912 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:48:19.048837 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-t6p2k_8b7836b7-c6dd-413a-bdc5-f5e98e9dcac3/main/0.log" Nov 04 05:48:19.604480 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:48:19.604455 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-ggvl5_e4282bcb-4054-4202-8513-1392d158ae81/main/0.log" Nov 04 05:48:28.341218 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:48:28.341176 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:48:58.341086 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:48:58.341007 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:49:28.340920 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:49:28.340884 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:49:30.401257 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:30.401230 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nk2gk_26ae8a36-06bf-414e-b2bd-325033800b72/global-pull-secret-syncer/0.log" Nov 04 05:49:31.053856 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:31.053830 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-5zj4b_523aff5c-ac2b-43be-8e13-792f44b2c14e/connection-checker/0.log" Nov 04 05:49:31.522810 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:31.522785 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-h5ccq_e10001da-2e9e-4b80-b6e0-0f1fca3df854/connection-checker/0.log" Nov 04 05:49:31.980766 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:31.980728 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-pw5zp_2e2ba652-6141-4984-8384-51f9720a02ef/connection-checker/0.log" Nov 04 05:49:33.138820 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:33.138794 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x27vz_1a566754-7023-408d-bb6a-3eb984ffa7e3/konnectivity-agent/0.log" Nov 04 05:49:34.544181 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:34.544153 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-143.us-west-2.compute.internal_9d7645ed4719bb9894aec65f339b9ca6/haproxy/0.log" Nov 04 05:49:40.036255 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036221 4859 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t"] Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036684 4859 cpu_manager.go:414] "containerMap: removing container" podUID="8018ad9e-115b-47b3-9475-e6b06e8de89f" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036696 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8018ad9e-115b-47b3-9475-e6b06e8de89f" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036709 4859 cpu_manager.go:414] "containerMap: removing container" podUID="3d2b29d5-afab-4bfd-878f-615e74d45cfa" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036715 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d2b29d5-afab-4bfd-878f-615e74d45cfa" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036732 4859 cpu_manager.go:414] "containerMap: removing container" podUID="a6f3c605-6d79-4057-9203-c965c9084fbe" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036738 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6f3c605-6d79-4057-9203-c965c9084fbe" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036747 4859 cpu_manager.go:414] "containerMap: removing container" podUID="7e5b156e-c723-400a-b76e-4d914ad58712" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036751 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e5b156e-c723-400a-b76e-4d914ad58712" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036771 4859 cpu_manager.go:414] "containerMap: removing container" podUID="2d4d34a1-0909-4690-8b21-e90d76afdfde" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036776 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d4d34a1-0909-4690-8b21-e90d76afdfde" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036787 4859 cpu_manager.go:414] "containerMap: removing container" podUID="6b3635c9-2a57-4ebb-a371-877a21946f46" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036791 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b3635c9-2a57-4ebb-a371-877a21946f46" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036798 4859 cpu_manager.go:414] "containerMap: removing container" podUID="1987453d-4e27-42c8-9737-a6689b817176" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036803 4859 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1987453d-4e27-42c8-9737-a6689b817176" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036856 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d2b29d5-afab-4bfd-878f-615e74d45cfa" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036866 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="a6f3c605-6d79-4057-9203-c965c9084fbe" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036872 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e5b156e-c723-400a-b76e-4d914ad58712" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036880 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d4d34a1-0909-4690-8b21-e90d76afdfde" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036887 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b3635c9-2a57-4ebb-a371-877a21946f46" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036893 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="1987453d-4e27-42c8-9737-a6689b817176" containerName="istio-proxy" Nov 04 05:49:40.038607 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.036899 4859 memory_manager.go:365] "RemoveStaleState removing state" podUID="8018ad9e-115b-47b3-9475-e6b06e8de89f" containerName="istio-proxy" Nov 04 05:49:40.039886 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.039871 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.099982 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.099960 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wtjmg\"/\"openshift-service-ca.crt\"" Nov 04 05:49:40.136814 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.136791 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6d8d241c-cbd2-41c6-9104-8683eecdbbaa-proc\") pod \"perf-node-gather-daemonset-r7r6t\" (UID: \"6d8d241c-cbd2-41c6-9104-8683eecdbbaa\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.136944 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.136834 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpvst\" (UniqueName: \"kubernetes.io/projected/6d8d241c-cbd2-41c6-9104-8683eecdbbaa-kube-api-access-fpvst\") pod \"perf-node-gather-daemonset-r7r6t\" (UID: \"6d8d241c-cbd2-41c6-9104-8683eecdbbaa\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.137021 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.136999 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6d8d241c-cbd2-41c6-9104-8683eecdbbaa-podres\") pod \"perf-node-gather-daemonset-r7r6t\" (UID: \"6d8d241c-cbd2-41c6-9104-8683eecdbbaa\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.137065 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.137038 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d8d241c-cbd2-41c6-9104-8683eecdbbaa-lib-modules\") pod \"perf-node-gather-daemonset-r7r6t\" (UID: \"6d8d241c-cbd2-41c6-9104-8683eecdbbaa\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.137106 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.137062 4859 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d8d241c-cbd2-41c6-9104-8683eecdbbaa-sys\") pod \"perf-node-gather-daemonset-r7r6t\" (UID: \"6d8d241c-cbd2-41c6-9104-8683eecdbbaa\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.155429 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.155400 4859 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wtjmg\"/\"kube-root-ca.crt\"" Nov 04 05:49:40.155554 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.155473 4859 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wtjmg\"/\"default-dockercfg-l9jq4\"" Nov 04 05:49:40.157455 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.157430 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t"] Nov 04 05:49:40.237846 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.237824 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpvst\" (UniqueName: \"kubernetes.io/projected/6d8d241c-cbd2-41c6-9104-8683eecdbbaa-kube-api-access-fpvst\") pod \"perf-node-gather-daemonset-r7r6t\" (UID: \"6d8d241c-cbd2-41c6-9104-8683eecdbbaa\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.237949 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.237923 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6d8d241c-cbd2-41c6-9104-8683eecdbbaa-podres\") pod \"perf-node-gather-daemonset-r7r6t\" (UID: \"6d8d241c-cbd2-41c6-9104-8683eecdbbaa\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.237995 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.237952 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d8d241c-cbd2-41c6-9104-8683eecdbbaa-lib-modules\") pod \"perf-node-gather-daemonset-r7r6t\" (UID: \"6d8d241c-cbd2-41c6-9104-8683eecdbbaa\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.237995 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.237983 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d8d241c-cbd2-41c6-9104-8683eecdbbaa-sys\") pod \"perf-node-gather-daemonset-r7r6t\" (UID: \"6d8d241c-cbd2-41c6-9104-8683eecdbbaa\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.238108 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.238007 4859 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6d8d241c-cbd2-41c6-9104-8683eecdbbaa-proc\") pod \"perf-node-gather-daemonset-r7r6t\" (UID: \"6d8d241c-cbd2-41c6-9104-8683eecdbbaa\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.238108 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.238048 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6d8d241c-cbd2-41c6-9104-8683eecdbbaa-podres\") pod \"perf-node-gather-daemonset-r7r6t\" (UID: \"6d8d241c-cbd2-41c6-9104-8683eecdbbaa\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.238108 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.238068 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d8d241c-cbd2-41c6-9104-8683eecdbbaa-sys\") pod \"perf-node-gather-daemonset-r7r6t\" (UID: \"6d8d241c-cbd2-41c6-9104-8683eecdbbaa\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.238108 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.238078 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d8d241c-cbd2-41c6-9104-8683eecdbbaa-lib-modules\") pod \"perf-node-gather-daemonset-r7r6t\" (UID: \"6d8d241c-cbd2-41c6-9104-8683eecdbbaa\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.238108 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.238102 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6d8d241c-cbd2-41c6-9104-8683eecdbbaa-proc\") pod \"perf-node-gather-daemonset-r7r6t\" (UID: \"6d8d241c-cbd2-41c6-9104-8683eecdbbaa\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.304349 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.304287 4859 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpvst\" (UniqueName: \"kubernetes.io/projected/6d8d241c-cbd2-41c6-9104-8683eecdbbaa-kube-api-access-fpvst\") pod \"perf-node-gather-daemonset-r7r6t\" (UID: \"6d8d241c-cbd2-41c6-9104-8683eecdbbaa\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.353164 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.353142 4859 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:40.737261 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:40.737231 4859 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t"] Nov 04 05:49:40.742179 ip-10-0-133-143 kubenswrapper[4859]: W1104 05:49:40.742142 4859 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6d8d241c_cbd2_41c6_9104_8683eecdbbaa.slice/crio-c8fd8877f94a2a1e005ac0f3b5e37ec606c711052fa6604c6e6d975b8c25c9ce WatchSource:0}: Error finding container c8fd8877f94a2a1e005ac0f3b5e37ec606c711052fa6604c6e6d975b8c25c9ce: Status 404 returned error can't find the container with id c8fd8877f94a2a1e005ac0f3b5e37ec606c711052fa6604c6e6d975b8c25c9ce Nov 04 05:49:41.569800 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:41.569765 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" event={"ID":"6d8d241c-cbd2-41c6-9104-8683eecdbbaa","Type":"ContainerStarted","Data":"9023911cff333e6659c106cc1627114094fe6c144de1eeb6f7d0b248948edf38"} Nov 04 05:49:41.570185 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:41.569813 4859 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" event={"ID":"6d8d241c-cbd2-41c6-9104-8683eecdbbaa","Type":"ContainerStarted","Data":"c8fd8877f94a2a1e005ac0f3b5e37ec606c711052fa6604c6e6d975b8c25c9ce"} Nov 04 05:49:41.570185 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:41.569843 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:41.695912 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:41.695865 4859 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" podStartSLOduration=2.695853567 podStartE2EDuration="2.695853567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:49:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:49:41.69534742 +0000 UTC m=+1351.696983144" watchObservedRunningTime="2026-11-04 05:49:41.695853567 +0000 UTC m=+1351.697489291" Nov 04 05:49:46.863436 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:46.863396 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bcbdb63b-fd44-4aaf-9dfe-769850a938ea/alertmanager/0.log" Nov 04 05:49:47.328810 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:47.328781 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bcbdb63b-fd44-4aaf-9dfe-769850a938ea/config-reloader/0.log" Nov 04 05:49:47.584181 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:47.584104 4859 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-r7r6t" Nov 04 05:49:47.791938 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:47.791913 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bcbdb63b-fd44-4aaf-9dfe-769850a938ea/kube-rbac-proxy-web/0.log" Nov 04 05:49:48.255263 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:48.255232 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bcbdb63b-fd44-4aaf-9dfe-769850a938ea/kube-rbac-proxy/0.log" Nov 04 05:49:48.718570 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:48.718523 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bcbdb63b-fd44-4aaf-9dfe-769850a938ea/kube-rbac-proxy-metric/0.log" Nov 04 05:49:49.188648 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:49.188623 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bcbdb63b-fd44-4aaf-9dfe-769850a938ea/prom-label-proxy/0.log" Nov 04 05:49:49.657584 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:49.657513 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bcbdb63b-fd44-4aaf-9dfe-769850a938ea/init-config-reloader/0.log" Nov 04 05:49:50.155395 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:50.155367 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-hcsmg_191dfb71-401c-494a-b7bb-cbc6c3273430/cluster-monitoring-operator/0.log" Nov 04 05:49:54.376460 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:54.376430 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bbm2w_ffdcf7c5-6d29-4e42-b8ac-6495001fea2d/node-exporter/0.log" Nov 04 05:49:54.838240 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:54.838207 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bbm2w_ffdcf7c5-6d29-4e42-b8ac-6495001fea2d/kube-rbac-proxy/0.log" Nov 04 05:49:55.301125 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:49:55.301101 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bbm2w_ffdcf7c5-6d29-4e42-b8ac-6495001fea2d/init-textfile/0.log" Nov 04 05:49:58.341109 ip-10-0-133-143 kubenswrapper[4859]: E1104 05:49:58.341079 4859 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-xfgjz" containerName="extractor" Nov 04 05:50:03.256574 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:50:03.256540 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cf64454f5-lp7q5_7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544/telemeter-client/0.log" Nov 04 05:50:03.722061 ip-10-0-133-143 kubenswrapper[4859]: I1104 05:50:03.722028 4859 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cf64454f5-lp7q5_7a1d3ac9-d6ba-4027-aee2-77fc2ff9e544/reload/0.log"