Nov 04 05:25:27.885233 ip-10-0-128-126 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Nov 04 05:25:27.885242 ip-10-0-128-126 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Nov 04 05:25:27.885249 ip-10-0-128-126 systemd[1]: kubelet.service: Failed with result 'resources'. Nov 04 05:25:27.885465 ip-10-0-128-126 systemd[1]: Failed to start Kubernetes Kubelet. Nov 04 05:25:38.004712 ip-10-0-128-126 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Nov 04 05:25:38.004727 ip-10-0-128-126 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot bf1c9f868385409fa364d5ff5618fe4b -- Nov 04 05:28:02.186508 ip-10-0-128-126 systemd[1]: Starting Kubernetes Kubelet... Nov 04 05:28:02.613564 ip-10-0-128-126 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:28:02.613564 ip-10-0-128-126 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 05:28:02.613564 ip-10-0-128-126 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:28:02.613564 ip-10-0-128-126 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:28:02.617870 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617790 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:28:02.617870 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617871 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617876 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617879 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617883 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617886 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617889 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617891 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617894 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617897 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617900 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617902 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617905 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617908 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617911 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617914 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617917 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617920 2579 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617922 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617925 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:28:02.617937 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617928 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617931 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617933 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617936 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617939 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617942 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617945 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617953 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617956 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617959 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617962 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617964 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617967 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617969 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617972 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617975 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617978 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617980 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617983 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:28:02.618452 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617985 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617988 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617991 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617993 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617996 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.617999 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618001 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618004 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618007 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618010 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618013 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618015 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618018 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618020 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618023 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618026 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618029 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618032 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618035 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:28:02.618901 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618037 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618040 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618043 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618045 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618049 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618051 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618054 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618056 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618067 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618070 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618073 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618076 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618078 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618081 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618084 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618086 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618089 2579 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618091 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618094 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618097 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618099 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:28:02.619399 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618103 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618106 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618109 2579 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618112 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618115 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618117 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618121 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618124 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618127 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618130 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618133 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618135 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618138 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618141 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618143 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618146 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618149 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618151 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618154 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:28:02.619980 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618157 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618162 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618165 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618169 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618172 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618175 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618178 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618180 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.618998 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619005 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619009 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619012 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619015 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619017 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619020 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619023 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619026 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619029 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619032 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619034 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:28:02.620448 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619039 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619042 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619045 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619047 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619050 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619053 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619056 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619058 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619061 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619064 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619066 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619069 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619072 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619074 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619077 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619080 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619083 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619086 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619088 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:28:02.620930 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619091 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619093 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619096 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619099 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619101 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619105 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619108 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619110 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619113 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619116 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619119 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619122 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619124 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619129 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619133 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619135 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619138 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619141 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619143 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:28:02.621389 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619146 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619149 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619151 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619154 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619157 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619160 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619163 2579 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619165 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619168 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619171 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619173 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619176 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619179 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619181 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619184 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619186 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619189 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619191 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619194 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619197 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:28:02.621871 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619199 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619202 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619204 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619208 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619210 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619213 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619216 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619219 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619238 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619242 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619245 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619247 2579 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619250 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619253 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619256 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619258 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619261 2579 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619264 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619267 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619269 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:28:02.622376 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619272 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619275 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619277 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619280 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619282 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619285 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619288 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619291 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619294 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619298 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619304 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619307 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619310 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619313 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619315 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619318 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619819 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619824 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619827 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:28:02.622849 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619829 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619832 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619835 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619837 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619840 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619843 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619846 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619849 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619851 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619854 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619857 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619859 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619862 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619866 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619869 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619874 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619876 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619879 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619881 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619884 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:28:02.623323 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619887 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619890 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619892 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619895 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619898 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619900 2579 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619903 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619908 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619910 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619914 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619917 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619919 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619922 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619925 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619927 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619930 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619933 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619935 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619938 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:28:02.623793 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619940 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619943 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619946 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619948 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619951 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619953 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619956 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619959 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619963 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619966 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619968 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619971 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619974 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619976 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619979 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619982 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619984 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619987 2579 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619990 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:28:02.624255 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619993 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.619996 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620000 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620002 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620007 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620011 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620015 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620019 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620022 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620025 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620027 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620030 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620035 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620039 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620042 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620045 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620048 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620051 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:28:02.624700 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620054 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620057 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620060 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620065 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620067 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620070 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620073 2579 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620075 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620078 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620081 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620083 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620086 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620088 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620091 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620093 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620096 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620099 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620102 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620104 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620107 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:28:02.625137 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620110 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620113 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620115 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620118 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620120 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620123 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.620126 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621125 2579 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621135 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621142 2579 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621147 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621152 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621156 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621161 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621165 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621170 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621173 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621177 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621180 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621183 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621186 2579 flags.go:64] FLAG: --cgroup-root="" Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621189 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621193 2579 flags.go:64] FLAG: --client-ca-file="" Nov 04 05:28:02.625623 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621196 2579 flags.go:64] FLAG: --cloud-provider="external" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621199 2579 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621203 2579 flags.go:64] FLAG: --cluster-domain="" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621206 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621210 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621213 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621217 2579 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621220 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621237 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621242 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621245 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621248 2579 flags.go:64] FLAG: --contention-profiling="false" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621251 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621254 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621258 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621261 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621266 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621269 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621272 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621275 2579 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621278 2579 flags.go:64] FLAG: --enable-server="true" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621281 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621286 2579 flags.go:64] FLAG: --event-burst="100" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621290 2579 flags.go:64] FLAG: --event-qps="50" Nov 04 05:28:02.626177 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621294 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621298 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621301 2579 flags.go:64] FLAG: --eviction-hard="" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621305 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621308 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621311 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621315 2579 flags.go:64] FLAG: --eviction-soft="" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621318 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621321 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621324 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621327 2579 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621330 2579 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621333 2579 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621336 2579 flags.go:64] FLAG: --feature-gates="" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621341 2579 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621344 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621347 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621351 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621354 2579 flags.go:64] FLAG: --healthz-port="10248" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621357 2579 flags.go:64] FLAG: --help="false" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621360 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621364 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621367 2579 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 05:28:02.626811 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621370 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621374 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621377 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621380 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621383 2579 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621386 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621389 2579 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621392 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621396 2579 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621402 2579 flags.go:64] FLAG: --kube-reserved="" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621405 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621408 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621413 2579 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621416 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621419 2579 flags.go:64] FLAG: --lock-file="" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621422 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621425 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621428 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621434 2579 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621437 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621440 2579 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621443 2579 flags.go:64] FLAG: --logging-format="text" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621446 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621449 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 05:28:02.627398 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621452 2579 flags.go:64] FLAG: --manifest-url="" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621456 2579 flags.go:64] FLAG: --manifest-url-header="" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621461 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621464 2579 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621473 2579 flags.go:64] FLAG: --max-pods="110" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621477 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621480 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621483 2579 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621486 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621489 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621492 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621495 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621503 2579 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621521 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621525 2579 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621528 2579 flags.go:64] FLAG: --pod-cidr="" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621532 2579 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621535 2579 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621539 2579 flags.go:64] FLAG: --pods-per-core="0" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621542 2579 flags.go:64] FLAG: --port="10250" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621546 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621549 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-01a4a5a33d44c4be0" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621552 2579 flags.go:64] FLAG: --qos-reserved="" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621556 2579 flags.go:64] FLAG: --read-only-port="10255" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621559 2579 flags.go:64] FLAG: --register-node="true" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621561 2579 flags.go:64] FLAG: --register-with-taints="" Nov 04 05:28:02.627972 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621569 2579 flags.go:64] FLAG: --registry-burst="10" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621573 2579 flags.go:64] FLAG: --registry-qps="5" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621576 2579 flags.go:64] FLAG: --reserved-cpus="" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621579 2579 flags.go:64] FLAG: --reserved-memory="" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621583 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621586 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621590 2579 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621593 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621596 2579 flags.go:64] FLAG: --runonce="false" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621599 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621602 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621606 2579 flags.go:64] FLAG: --seccomp-default="false" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621609 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621612 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621615 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621618 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621621 2579 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621624 2579 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621627 2579 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621630 2579 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621633 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621636 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621639 2579 flags.go:64] FLAG: --system-cgroups="" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621642 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621648 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 05:28:02.628649 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621651 2579 flags.go:64] FLAG: --tls-cert-file="" Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621655 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621660 2579 flags.go:64] FLAG: --tls-min-version="" Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621662 2579 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621666 2579 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621669 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621672 2579 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621675 2579 flags.go:64] FLAG: --v="2" Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621679 2579 flags.go:64] FLAG: --version="false" Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621683 2579 flags.go:64] FLAG: --vmodule="" Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621687 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.621691 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621831 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621835 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621838 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621841 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621844 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621847 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621850 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621853 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621856 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621859 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:28:02.629768 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621861 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621864 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621867 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621869 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621872 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621875 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621878 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621881 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621884 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621887 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621891 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621896 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621899 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621902 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621905 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621908 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621911 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621914 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621916 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:28:02.630336 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621919 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621922 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621924 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621927 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621931 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621934 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621937 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621940 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621943 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621945 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621948 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621951 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621954 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621957 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621959 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621962 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621964 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621967 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621969 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:28:02.630788 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621972 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621975 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621978 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621980 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621983 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621987 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621990 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621992 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621995 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.621998 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622000 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622003 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622006 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622008 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622011 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622014 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622016 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622019 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622022 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:28:02.631374 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622024 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622027 2579 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622029 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622032 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622035 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622037 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622040 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622044 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622046 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622049 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622052 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622054 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622057 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622067 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622070 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622074 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622077 2579 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622080 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622084 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622086 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:28:02.631831 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622089 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622092 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622094 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622097 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622100 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622102 2579 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622105 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622108 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622110 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622113 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622116 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622118 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622121 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622124 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622126 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622129 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622132 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622134 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:28:02.632326 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622137 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:28:02.632750 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.622164 2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:28:02.632750 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622200 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:28:02.632750 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622205 2579 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:28:02.632750 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622208 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:28:02.632750 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622211 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:28:02.632750 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622213 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:28:02.632750 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622216 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:28:02.632750 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622219 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:28:02.632750 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622221 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:28:02.632750 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622236 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:28:02.632750 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622239 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:28:02.632750 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622243 2579 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:28:02.632750 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622246 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:28:02.632750 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622250 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:28:02.632750 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622252 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:28:02.632750 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622255 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622258 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622260 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622263 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622266 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622269 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622271 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622274 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622277 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622279 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622282 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622285 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622287 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622290 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622293 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622296 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622298 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622301 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622304 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622307 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:28:02.633135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622309 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622312 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622315 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622317 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622320 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622323 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622326 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622329 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622333 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622335 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622339 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622343 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622347 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622349 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622352 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622354 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622357 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622360 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622363 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:28:02.633618 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622365 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622368 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622371 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622374 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622379 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622383 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622386 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622389 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622392 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622395 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622397 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622400 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622404 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622406 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622409 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622412 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622414 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622417 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622420 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:28:02.634098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622423 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622426 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622430 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622432 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622440 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622443 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622446 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622449 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622451 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622454 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622457 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622459 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622462 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622465 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622468 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622471 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622473 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622476 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622478 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:28:02.634638 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622481 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622484 2579 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622486 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622489 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622492 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622494 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622497 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622500 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622503 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622506 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622509 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622512 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622514 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622519 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.622522 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.629584 2579 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.629601 2579 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.630312 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.630318 2579 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630376 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:28:02.635126 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630382 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630385 2579 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630389 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630394 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630398 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630402 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630405 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630408 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630411 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630414 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630417 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630420 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630423 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630425 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630428 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630431 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630433 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630436 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630439 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:28:02.635675 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630441 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630444 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630447 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630449 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630452 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630455 2579 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630457 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630460 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630463 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630468 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630471 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630474 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630477 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630479 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630482 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630485 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630487 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630490 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630493 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630495 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:28:02.636118 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630498 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630501 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630503 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630506 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630509 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630511 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630514 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630516 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630520 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630522 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630525 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630527 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630530 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630535 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630538 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630541 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630544 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630547 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630549 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:28:02.636613 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630552 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630555 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630558 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630561 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630564 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630567 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630569 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630572 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630575 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630577 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630580 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630583 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630585 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630588 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630591 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630593 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630596 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630598 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630601 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:28:02.637286 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630603 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630606 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630609 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630612 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630614 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630617 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630619 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630623 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630625 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630628 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630631 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630634 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630637 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630639 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630642 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630645 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630647 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630651 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630653 2579 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630656 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:28:02.638004 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630659 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:28:02.638592 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630661 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:28:02.638592 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630664 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:28:02.638592 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630666 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:28:02.638592 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630669 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:28:02.638592 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630671 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:28:02.638592 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630674 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:28:02.638592 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630677 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:28:02.638592 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.630703 2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:28:02.638592 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630842 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:28:02.638592 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630847 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:28:02.638592 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630849 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:28:02.638592 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630852 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:28:02.638592 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630855 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:28:02.638592 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630858 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:28:02.638592 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630860 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:28:02.638592 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630863 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630866 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630868 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630871 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630874 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630876 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630879 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630882 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630885 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630888 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630890 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630893 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630896 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630898 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630901 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630904 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630906 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630909 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630911 2579 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630915 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:28:02.639074 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630917 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630920 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630922 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630925 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630927 2579 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630930 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630933 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630935 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630938 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630941 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630943 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630946 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630948 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630951 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630954 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630956 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630959 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630961 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630964 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:28:02.639745 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630968 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630972 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630975 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630978 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630980 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630983 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630987 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630990 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630994 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.630997 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631000 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631003 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631005 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631008 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631011 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631013 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631016 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631018 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631021 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:28:02.640340 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631024 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631026 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631029 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631031 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631034 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631036 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631039 2579 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631042 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631044 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631047 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631049 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631052 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631055 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631058 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631060 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631063 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631065 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631068 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631070 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631073 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:28:02.640801 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631075 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631078 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631082 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631085 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631087 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631089 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631092 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631095 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631097 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631100 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631103 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631105 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631108 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631111 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631113 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631116 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631119 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631121 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631124 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:28:02.641300 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631126 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:28:02.641865 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631129 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:28:02.641865 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.631152 2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:28:02.641865 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631182 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:28:02.641865 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631185 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:28:02.641865 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631188 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:28:02.641865 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631191 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:28:02.641865 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631194 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:28:02.641865 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631196 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:28:02.641865 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631199 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:28:02.641865 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631202 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:28:02.641865 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631204 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:28:02.641865 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631207 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:28:02.641865 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631209 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:28:02.641865 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631212 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:28:02.641865 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631217 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:28:02.641865 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631220 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631237 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631241 2579 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631245 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631249 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631251 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631254 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631257 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631260 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631262 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631265 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631267 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631270 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631273 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631275 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631278 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631280 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631283 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631286 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631288 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:28:02.642343 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631290 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631293 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631295 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631298 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631300 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631303 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631306 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631308 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631311 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631313 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631316 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631320 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631323 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631326 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631329 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631331 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631334 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631336 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631339 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:28:02.642815 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631343 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631346 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631349 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631351 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631354 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631357 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631360 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631362 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631365 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631367 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631370 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631373 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631376 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631378 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631381 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631384 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631387 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631390 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631392 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:28:02.643282 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631395 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631398 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631400 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631403 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631405 2579 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631411 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631413 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631416 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631419 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631422 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631424 2579 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631427 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631430 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631433 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631435 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631438 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631440 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631443 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631446 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631448 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:28:02.643723 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631451 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631453 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631456 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631458 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631461 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631465 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631469 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631472 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631474 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631477 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631480 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631483 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631487 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631490 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:02.631493 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.631545 2579 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.633466 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.634422 2579 server.go:1014] "Starting client certificate rotation" Nov 04 05:28:02.644186 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.634529 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 05:28:02.644651 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.634578 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 05:28:02.644651 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.636371 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 05:28:02.644651 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.639928 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 05:28:02.651327 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.651306 2579 log.go:25] "Validated CRI v1 runtime API" Nov 04 05:28:02.654948 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.654931 2579 log.go:25] "Validated CRI v1 image API" Nov 04 05:28:02.655852 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.655824 2579 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 05:28:02.661853 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.661830 2579 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8d969107-3d79-4478-b37d-8eaec9420231:/dev/nvme0n1p4 c6586ec7-5431-4d80-92d2-d95ded28bf70:/dev/nvme0n1p3] Nov 04 05:28:02.661923 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.661849 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Nov 04 05:28:02.665590 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.665560 2579 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 05:28:02.667020 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.666901 2579 manager.go:217] Machine: {Timestamp:2026-11-04 05:28:02.665511414 +0000 UTC m=+0.353341033 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099358 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec224560ebdf1dc4588f1bbcd6e09ad9 SystemUUID:ec224560-ebdf-1dc4-588f-1bbcd6e09ad9 BootID:bf1c9f86-8385-409f-a364-d5ff5618fe4b Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:cf:85:a6:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:cf:85:a6:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:c9:4c:8e:e8:af Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 05:28:02.667020 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.667016 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 05:28:02.667203 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.667178 2579 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 05:28:02.670279 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.670256 2579 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 05:28:02.670460 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.670282 2579 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-126.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 05:28:02.670541 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.670465 2579 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 05:28:02.670541 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.670477 2579 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 05:28:02.670541 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.670503 2579 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 05:28:02.670541 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.670523 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 05:28:02.671374 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.671361 2579 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 05:28:02.675548 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.675534 2579 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 05:28:02.675715 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.675703 2579 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 05:28:02.675774 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.675743 2579 kubelet.go:493] "Attempting to sync node with API server" Nov 04 05:28:02.675774 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.675762 2579 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 05:28:02.675866 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.675776 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 05:28:02.675866 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.675790 2579 kubelet.go:405] "Adding apiserver pod source" Nov 04 05:28:02.675866 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.675802 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 05:28:02.676154 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.676141 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 05:28:02.678589 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.678572 2579 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 05:28:02.679435 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.679415 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-smrj7" Nov 04 05:28:02.684317 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.684299 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 05:28:02.685138 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.685117 2579 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 05:28:02.685257 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.685168 2579 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 05:28:02.685392 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.685380 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 05:28:02.685424 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.685401 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 05:28:02.685424 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.685412 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 05:28:02.685424 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.685421 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 05:28:02.685495 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.685430 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 05:28:02.685495 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.685439 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 05:28:02.685919 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.685906 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 05:28:02.685919 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.685920 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 05:28:02.686000 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.685928 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 05:28:02.686000 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.685934 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 05:28:02.686000 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.685943 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 05:28:02.686000 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.685953 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 05:28:02.686607 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.686587 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 05:28:02.686703 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.686611 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 05:28:02.686703 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.686654 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-smrj7" Nov 04 05:28:02.690297 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.690283 2579 server.go:1263] "Started kubelet" Nov 04 05:28:02.690661 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.690606 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 05:28:02.690715 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.690677 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 05:28:02.690770 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.690742 2579 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 05:28:02.691041 ip-10-0-128-126 systemd[1]: Started Kubernetes Kubelet. Nov 04 05:28:02.692522 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.692503 2579 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 05:28:02.693416 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.693399 2579 server.go:317] "Adding debug handlers to kubelet server" Nov 04 05:28:02.697455 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:02.697431 2579 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Nov 04 05:28:02.698200 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.698178 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 05:28:02.698600 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.698582 2579 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 05:28:02.698857 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:02.698838 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-126.us-east-2.compute.internal\" not found" Nov 04 05:28:02.699143 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.699127 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 05:28:02.699143 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.699136 2579 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 05:28:02.699293 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.699152 2579 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 05:28:02.699340 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.699295 2579 reconstruct.go:97] "Volume reconstruction finished" Nov 04 05:28:02.699340 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.699302 2579 reconciler.go:29] "Reconciler: start to sync state" Nov 04 05:28:02.699340 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.699316 2579 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:28:02.700203 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.700187 2579 factory.go:55] Registering systemd factory Nov 04 05:28:02.700203 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.700203 2579 factory.go:223] Registration of the systemd container factory successfully Nov 04 05:28:02.700432 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.700255 2579 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-128-126.us-east-2.compute.internal" not found Nov 04 05:28:02.700506 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.700478 2579 factory.go:153] Registering CRI-O factory Nov 04 05:28:02.700506 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.700491 2579 factory.go:223] Registration of the crio container factory successfully Nov 04 05:28:02.700579 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.700545 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 05:28:02.700579 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.700564 2579 factory.go:103] Registering Raw factory Nov 04 05:28:02.700579 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.700576 2579 manager.go:1199] Started watching for new ooms in manager Nov 04 05:28:02.701394 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.701380 2579 manager.go:319] Starting recovery of all containers Nov 04 05:28:02.701526 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.701508 2579 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:28:02.701992 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.701976 2579 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:28:02.710458 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:02.704606 2579 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-126.us-east-2.compute.internal\" not found" node="ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:02.713885 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.713865 2579 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-128-126.us-east-2.compute.internal" not found Nov 04 05:28:02.715060 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.715044 2579 manager.go:324] Recovery completed Nov 04 05:28:02.718917 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.718905 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:28:02.720625 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.720610 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-126.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:28:02.720698 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.720637 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-126.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:28:02.720698 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.720648 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-126.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:28:02.721217 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.721202 2579 cpu_manager.go:226] "Starting" policy="none" Nov 04 05:28:02.721217 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.721211 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:28:02.721217 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.721215 2579 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 05:28:02.721345 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.721247 2579 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 05:28:02.721861 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.721849 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-126.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:28:02.721901 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.721873 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-126.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:28:02.721901 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.721882 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-126.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:28:02.723390 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.723379 2579 policy_none.go:50] "Start" Nov 04 05:28:02.723425 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.723395 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 05:28:02.723425 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.723404 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 05:28:02.725352 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.725342 2579 policy_none.go:44] "Start" Nov 04 05:28:02.769083 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.769058 2579 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-128-126.us-east-2.compute.internal" not found Nov 04 05:28:02.799937 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:02.799917 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-126.us-east-2.compute.internal\" not found" Nov 04 05:28:02.812795 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.812759 2579 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 05:28:02.816787 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.816775 2579 manager.go:341] "Starting Device Plugin manager" Nov 04 05:28:02.816877 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:02.816864 2579 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 05:28:02.816877 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.816877 2579 server.go:87] "Starting device plugin registration server" Nov 04 05:28:02.817102 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.817093 2579 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 05:28:02.817135 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.817104 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 05:28:02.817266 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.817239 2579 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 05:28:02.817372 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.817334 2579 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 05:28:02.817372 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.817343 2579 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 05:28:02.817842 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:02.817825 2579 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 05:28:02.817934 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:02.817895 2579 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-126.us-east-2.compute.internal\" not found" Nov 04 05:28:02.917697 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.917676 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:28:02.919567 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.918710 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-126.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:28:02.919567 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.918770 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-126.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:28:02.919567 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.918783 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-126.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:28:02.919567 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.918811 2579 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:02.928096 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.928077 2579 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:02.974584 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.974435 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 05:28:02.975716 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.975690 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 05:28:02.976558 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.976543 2579 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 05:28:02.976654 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.976574 2579 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 05:28:02.976654 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:02.976617 2579 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Nov 04 05:28:02.983787 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:02.983771 2579 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:28:03.077595 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.077565 2579 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal"] Nov 04 05:28:03.079977 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.079961 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:03.081883 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.081863 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:03.101730 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.101705 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58571c34b4dbebe904f2b5a148e16c40-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"58571c34b4dbebe904f2b5a148e16c40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:03.101806 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.101734 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58571c34b4dbebe904f2b5a148e16c40-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"58571c34b4dbebe904f2b5a148e16c40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:03.101806 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.101757 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fb1cdcff7ca15ea5212ef7ed52c2b8e3-config\") pod \"kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"fb1cdcff7ca15ea5212ef7ed52c2b8e3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:03.110609 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.110594 2579 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:03.113262 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.113244 2579 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:03.120838 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.120825 2579 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:28:03.123860 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.123848 2579 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:28:03.202423 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.202348 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58571c34b4dbebe904f2b5a148e16c40-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"58571c34b4dbebe904f2b5a148e16c40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:03.202423 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.202386 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58571c34b4dbebe904f2b5a148e16c40-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"58571c34b4dbebe904f2b5a148e16c40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:03.202423 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.202418 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fb1cdcff7ca15ea5212ef7ed52c2b8e3-config\") pod \"kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"fb1cdcff7ca15ea5212ef7ed52c2b8e3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:03.202594 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.202474 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58571c34b4dbebe904f2b5a148e16c40-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"58571c34b4dbebe904f2b5a148e16c40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:03.202594 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.202477 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fb1cdcff7ca15ea5212ef7ed52c2b8e3-config\") pod \"kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"fb1cdcff7ca15ea5212ef7ed52c2b8e3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:03.202594 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.202477 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58571c34b4dbebe904f2b5a148e16c40-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"58571c34b4dbebe904f2b5a148e16c40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:03.425903 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.425840 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:03.426997 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.426975 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:28:03.634679 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.634588 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Nov 04 05:28:03.635307 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.634751 2579 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 04 05:28:03.635307 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.634798 2579 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 04 05:28:03.635307 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.634817 2579 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 04 05:28:03.677295 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.677267 2579 apiserver.go:52] "Watching apiserver" Nov 04 05:28:03.685308 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.685284 2579 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 05:28:03.685593 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.685575 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-fvxf5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal","openshift-network-diagnostics/network-check-target-mq5nl","openshift-network-operator/iptables-alerter-m77gq","kube-system/konnectivity-agent-m66r7","openshift-multus/multus-hk726","openshift-multus/network-metrics-daemon-brh9c","openshift-cluster-node-tuning-operator/tuned-gbms4","openshift-image-registry/node-ca-rkg7g","kube-system/global-pull-secret-syncer-69rl9","openshift-multus/multus-additional-cni-plugins-z9djg","openshift-ovn-kubernetes/ovnkube-node-slrx2","kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx"] Nov 04 05:28:03.688183 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.688152 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:28:03.690382 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.690365 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:03.690506 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:03.690450 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:03.690642 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.690602 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 05:23:03 +0000 UTC" deadline="2028-04-06 14:24:55.100973653 +0000 UTC" Nov 04 05:28:03.690642 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.690640 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12464h56m51.410336041s" Nov 04 05:28:03.692758 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.692570 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qlwsz\"" Nov 04 05:28:03.692758 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.692721 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 05:28:03.692931 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.692893 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:28:03.693342 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.693324 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 05:28:03.695151 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.695119 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:28:03.697994 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.697975 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hk726" Nov 04 05:28:03.698283 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.698262 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Nov 04 05:28:03.700192 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.700165 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:03.700284 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:03.700263 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:03.702397 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.702376 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.704766 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.704750 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:28:03.705473 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.705457 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-socket-dir-parent\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.705543 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.705481 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-cni-bin\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.705543 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.705499 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc4df\" (UniqueName: \"kubernetes.io/projected/f7b4c728-a80c-403d-88d9-af4fb0835993-kube-api-access-tc4df\") pod \"iptables-alerter-m77gq\" (UID: \"f7b4c728-a80c-403d-88d9-af4fb0835993\") " pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:28:03.705543 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.705515 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/915dd778-3692-4f41-83d3-cc5d4e73d615-agent-certs\") pod \"konnectivity-agent-m66r7\" (UID: \"915dd778-3692-4f41-83d3-cc5d4e73d615\") " pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:28:03.705543 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.705534 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-k8s-cni-cncf-io\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.705757 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.705605 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-etc-kubernetes\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.705757 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.705687 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7b4c728-a80c-403d-88d9-af4fb0835993-host-slash\") pod \"iptables-alerter-m77gq\" (UID: \"f7b4c728-a80c-403d-88d9-af4fb0835993\") " pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:28:03.705757 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.705735 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-multus-certs\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.705881 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.705769 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzqnm\" (UniqueName: \"kubernetes.io/projected/7239836a-64ce-4847-91f4-9360cd652ea6-kube-api-access-pzqnm\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.705881 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.705820 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-conf-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.705881 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.705874 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/80902534-76da-4e9d-8157-ce481e71f2f8-hosts-file\") pod \"node-resolver-fvxf5\" (UID: \"80902534-76da-4e9d-8157-ce481e71f2f8\") " pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:28:03.705997 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.705902 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80902534-76da-4e9d-8157-ce481e71f2f8-tmp-dir\") pod \"node-resolver-fvxf5\" (UID: \"80902534-76da-4e9d-8157-ce481e71f2f8\") " pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:28:03.705997 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.705930 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/915dd778-3692-4f41-83d3-cc5d4e73d615-konnectivity-ca\") pod \"konnectivity-agent-m66r7\" (UID: \"915dd778-3692-4f41-83d3-cc5d4e73d615\") " pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:28:03.705997 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.705952 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-system-cni-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.705997 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.705980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:03.706143 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.706005 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg65k\" (UniqueName: \"kubernetes.io/projected/efbcf64e-3ade-4343-a9c8-bf945b141a02-kube-api-access-kg65k\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:03.706143 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.706036 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghglb\" (UniqueName: \"kubernetes.io/projected/80902534-76da-4e9d-8157-ce481e71f2f8-kube-api-access-ghglb\") pod \"node-resolver-fvxf5\" (UID: \"80902534-76da-4e9d-8157-ce481e71f2f8\") " pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:28:03.706143 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.706056 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4b8q\" (UniqueName: \"kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q\") pod \"network-check-target-mq5nl\" (UID: \"47be26f2-143e-4b49-824a-415e60f6140b\") " pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:03.706143 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.706089 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-cnibin\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.706143 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.706105 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-netns\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.706143 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.706131 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-cni-multus\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.706418 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.706148 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-cni-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.706418 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.706169 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7239836a-64ce-4847-91f4-9360cd652ea6-cni-binary-copy\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.706418 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.706184 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-hostroot\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.706418 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.706201 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7239836a-64ce-4847-91f4-9360cd652ea6-multus-daemon-config\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.706418 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.706223 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7b4c728-a80c-403d-88d9-af4fb0835993-iptables-alerter-script\") pod \"iptables-alerter-m77gq\" (UID: \"f7b4c728-a80c-403d-88d9-af4fb0835993\") " pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:28:03.706418 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.706267 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-kubelet\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.706996 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.706982 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:03.707061 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:03.707043 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:03.709295 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.709277 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.712261 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.712243 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.714879 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.714789 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.731774 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.731751 2579 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 05:28:03.731855 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.731822 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wxvvr" Nov 04 05:28:03.741223 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.741207 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wxvvr" Nov 04 05:28:03.799792 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.799772 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 05:28:03.801056 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801022 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 05:28:03.801056 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801050 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xrb2c\"" Nov 04 05:28:03.801861 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801794 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 05:28:03.801861 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801792 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801861 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801836 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801836 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801835 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801880 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-67qcf\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801930 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801955 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801966 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801976 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-4fjpb\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801986 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801968 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801999 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.802007 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.802012 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.802018 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.802016 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.801988 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 05:28:03.802031 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.802034 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-m8rqh\"" Nov 04 05:28:03.802584 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.802053 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 05:28:03.802584 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.802077 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7x6ks\"" Nov 04 05:28:03.802584 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.802152 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-chnwp\"" Nov 04 05:28:03.802584 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.802279 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rpj2j\"" Nov 04 05:28:03.802584 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.802288 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 05:28:03.802584 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.802423 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 05:28:03.802584 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.802432 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fwwlp\"" Nov 04 05:28:03.803172 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.803157 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 05:28:03.803172 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.803167 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 05:28:03.803501 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.803488 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 05:28:03.803538 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.803531 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 05:28:03.806802 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.806786 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7239836a-64ce-4847-91f4-9360cd652ea6-multus-daemon-config\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.806892 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.806814 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-ovn\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.806892 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.806832 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-cni-netd\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.806892 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.806847 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-modprobe-d\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.806892 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.806863 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysconfig\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.807012 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.806910 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/343f8865-3015-4715-960c-a2e1418c6f7e-tmp\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.807012 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.806960 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-system-cni-dir\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.807012 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.806989 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-socket-dir-parent\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.807102 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807020 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-cni-bin\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.807135 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807124 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysctl-conf\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.807170 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807139 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-socket-dir-parent\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.807170 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807148 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-cni-bin\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.807229 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807180 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-host\") pod \"node-ca-rkg7g\" (UID: \"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01\") " pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:28:03.807229 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807198 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-cni-binary-copy\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.807310 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807282 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.807348 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807310 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.807348 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807337 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-k8s-cni-cncf-io\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.807410 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807391 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-k8s-cni-cncf-io\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.807441 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807417 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-etc-kubernetes\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.807473 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807439 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/379bdc95-36d7-47f2-9e17-a96e51da6495-kubelet-config\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:03.807473 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807460 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-kubernetes\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.807473 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807461 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-etc-kubernetes\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.807583 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807475 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-etc-selinux\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.807583 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807504 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-multus-certs\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.807583 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807530 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzqnm\" (UniqueName: \"kubernetes.io/projected/7239836a-64ce-4847-91f4-9360cd652ea6-kube-api-access-pzqnm\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.807583 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807536 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-multus-certs\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.807583 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807566 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-kubelet\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.807733 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807591 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-sys\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.807733 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807680 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/343f8865-3015-4715-960c-a2e1418c6f7e-etc-tuned\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.807733 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807706 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-socket-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.807854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807734 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-device-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.807854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807765 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-node-log\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.807854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807789 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80902534-76da-4e9d-8157-ce481e71f2f8-tmp-dir\") pod \"node-resolver-fvxf5\" (UID: \"80902534-76da-4e9d-8157-ce481e71f2f8\") " pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:28:03.807854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807807 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-system-cni-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.808071 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807851 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:03.808071 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807887 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.808071 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807919 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghglb\" (UniqueName: \"kubernetes.io/projected/80902534-76da-4e9d-8157-ce481e71f2f8-kube-api-access-ghglb\") pod \"node-resolver-fvxf5\" (UID: \"80902534-76da-4e9d-8157-ce481e71f2f8\") " pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:28:03.808071 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807931 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-system-cni-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.808071 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-cnibin\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.808071 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807974 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-netns\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.808071 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:03.808000 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:28:03.808071 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808021 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7239836a-64ce-4847-91f4-9360cd652ea6-multus-daemon-config\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.808071 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808059 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-netns\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.808498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.807999 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-slash\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.808498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808075 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-cnibin\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.808498 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:03.808109 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs podName:efbcf64e-3ade-4343-a9c8-bf945b141a02 nodeName:}" failed. No retries permitted until 2026-11-04 05:28:04.308038846 +0000 UTC m=+1.995868477 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs") pod "network-metrics-daemon-brh9c" (UID: "efbcf64e-3ade-4343-a9c8-bf945b141a02") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:28:03.808498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808118 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80902534-76da-4e9d-8157-ce481e71f2f8-tmp-dir\") pod \"node-resolver-fvxf5\" (UID: \"80902534-76da-4e9d-8157-ce481e71f2f8\") " pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:28:03.808498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808136 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-run-netns\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.808498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808175 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-etc-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.808498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808201 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-serviceca\") pod \"node-ca-rkg7g\" (UID: \"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01\") " pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:28:03.808498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808247 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-cni-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.808498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808285 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7239836a-64ce-4847-91f4-9360cd652ea6-cni-binary-copy\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.808498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808325 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-cni-bin\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.808498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808343 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gztg\" (UniqueName: \"kubernetes.io/projected/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-kube-api-access-5gztg\") pod \"node-ca-rkg7g\" (UID: \"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01\") " pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:28:03.808498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808362 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-cnibin\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.808498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808363 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-cni-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.808498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808405 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-systemd-units\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.808498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808438 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-var-lib-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.808498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808465 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjjl9\" (UniqueName: \"kubernetes.io/projected/82660df8-f278-4d7c-b02a-149527e30d27-kube-api-access-pjjl9\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808503 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7b4c728-a80c-403d-88d9-af4fb0835993-iptables-alerter-script\") pod \"iptables-alerter-m77gq\" (UID: \"f7b4c728-a80c-403d-88d9-af4fb0835993\") " pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808553 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-kubelet\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808615 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-kubelet\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808627 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808651 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-env-overrides\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808674 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-sys-fs\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808700 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-systemd\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808776 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-host\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808803 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc4df\" (UniqueName: \"kubernetes.io/projected/f7b4c728-a80c-403d-88d9-af4fb0835993-kube-api-access-tc4df\") pod \"iptables-alerter-m77gq\" (UID: \"f7b4c728-a80c-403d-88d9-af4fb0835993\") " pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.808799 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7239836a-64ce-4847-91f4-9360cd652ea6-cni-binary-copy\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809000 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/915dd778-3692-4f41-83d3-cc5d4e73d615-agent-certs\") pod \"konnectivity-agent-m66r7\" (UID: \"915dd778-3692-4f41-83d3-cc5d4e73d615\") " pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809037 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/82660df8-f278-4d7c-b02a-149527e30d27-ovn-node-metrics-cert\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809002 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7b4c728-a80c-403d-88d9-af4fb0835993-iptables-alerter-script\") pod \"iptables-alerter-m77gq\" (UID: \"f7b4c728-a80c-403d-88d9-af4fb0835993\") " pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809076 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-ovnkube-script-lib\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809101 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-var-lib-kubelet\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809116 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6rf9\" (UniqueName: \"kubernetes.io/projected/343f8865-3015-4715-960c-a2e1418c6f7e-kube-api-access-x6rf9\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.809126 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809132 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809151 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7b4c728-a80c-403d-88d9-af4fb0835993-host-slash\") pod \"iptables-alerter-m77gq\" (UID: \"f7b4c728-a80c-403d-88d9-af4fb0835993\") " pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809194 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809214 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7b4c728-a80c-403d-88d9-af4fb0835993-host-slash\") pod \"iptables-alerter-m77gq\" (UID: \"f7b4c728-a80c-403d-88d9-af4fb0835993\") " pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809243 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysctl-d\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809272 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809326 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-log-socket\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809326 2579 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809373 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-ovnkube-config\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809404 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-conf-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809444 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/80902534-76da-4e9d-8157-ce481e71f2f8-hosts-file\") pod \"node-resolver-fvxf5\" (UID: \"80902534-76da-4e9d-8157-ce481e71f2f8\") " pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809453 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-conf-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809466 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/915dd778-3692-4f41-83d3-cc5d4e73d615-konnectivity-ca\") pod \"konnectivity-agent-m66r7\" (UID: \"915dd778-3692-4f41-83d3-cc5d4e73d615\") " pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809492 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg65k\" (UniqueName: \"kubernetes.io/projected/efbcf64e-3ade-4343-a9c8-bf945b141a02-kube-api-access-kg65k\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809506 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/80902534-76da-4e9d-8157-ce481e71f2f8-hosts-file\") pod \"node-resolver-fvxf5\" (UID: \"80902534-76da-4e9d-8157-ce481e71f2f8\") " pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809517 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-run-ovn-kubernetes\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809547 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4b8q\" (UniqueName: \"kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q\") pod \"network-check-target-mq5nl\" (UID: \"47be26f2-143e-4b49-824a-415e60f6140b\") " pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:03.809823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809597 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-cni-multus\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.810401 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809628 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-systemd\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.810401 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809653 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-run\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.810401 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809676 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-cni-multus\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.810401 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809679 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-registration-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.810401 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809738 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/379bdc95-36d7-47f2-9e17-a96e51da6495-dbus\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:03.810401 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809759 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-lib-modules\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.810401 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809774 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvb55\" (UniqueName: \"kubernetes.io/projected/f4a51628-0aab-4614-a4ed-11f184521cd4-kube-api-access-nvb55\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.810401 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809827 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc46j\" (UniqueName: \"kubernetes.io/projected/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-kube-api-access-rc46j\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.810401 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809859 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-hostroot\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.810401 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809961 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-hostroot\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.810401 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.809992 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/915dd778-3692-4f41-83d3-cc5d4e73d615-konnectivity-ca\") pod \"konnectivity-agent-m66r7\" (UID: \"915dd778-3692-4f41-83d3-cc5d4e73d615\") " pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:28:03.813443 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.813427 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/915dd778-3692-4f41-83d3-cc5d4e73d615-agent-certs\") pod \"konnectivity-agent-m66r7\" (UID: \"915dd778-3692-4f41-83d3-cc5d4e73d615\") " pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:28:03.816539 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:03.816516 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:28:03.816657 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:03.816543 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:28:03.816657 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:03.816557 2579 projected.go:196] Error preparing data for projected volume kube-api-access-h4b8q for pod openshift-network-diagnostics/network-check-target-mq5nl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:28:03.816657 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:03.816633 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q podName:47be26f2-143e-4b49-824a-415e60f6140b nodeName:}" failed. No retries permitted until 2026-11-04 05:28:04.316614833 +0000 UTC m=+2.004444462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-h4b8q" (UniqueName: "kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q") pod "network-check-target-mq5nl" (UID: "47be26f2-143e-4b49-824a-415e60f6140b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:28:03.818440 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.818239 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc4df\" (UniqueName: \"kubernetes.io/projected/f7b4c728-a80c-403d-88d9-af4fb0835993-kube-api-access-tc4df\") pod \"iptables-alerter-m77gq\" (UID: \"f7b4c728-a80c-403d-88d9-af4fb0835993\") " pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:28:03.818511 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.818493 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghglb\" (UniqueName: \"kubernetes.io/projected/80902534-76da-4e9d-8157-ce481e71f2f8-kube-api-access-ghglb\") pod \"node-resolver-fvxf5\" (UID: \"80902534-76da-4e9d-8157-ce481e71f2f8\") " pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:28:03.818622 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.818601 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzqnm\" (UniqueName: \"kubernetes.io/projected/7239836a-64ce-4847-91f4-9360cd652ea6-kube-api-access-pzqnm\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:28:03.819769 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.819749 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg65k\" (UniqueName: \"kubernetes.io/projected/efbcf64e-3ade-4343-a9c8-bf945b141a02-kube-api-access-kg65k\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:03.910474 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.910388 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysctl-conf\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.910721 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.910687 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-host\") pod \"node-ca-rkg7g\" (UID: \"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01\") " pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:28:03.910911 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.910882 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-cni-binary-copy\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.910985 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.910919 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysctl-conf\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.910985 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.910929 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.910985 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.910839 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-host\") pod \"node-ca-rkg7g\" (UID: \"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01\") " pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:28:03.911169 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911008 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.911169 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911035 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/379bdc95-36d7-47f2-9e17-a96e51da6495-kubelet-config\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:03.911169 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911057 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-kubernetes\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.911169 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911087 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-etc-selinux\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.911169 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911129 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-kubelet\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.911459 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911176 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-sys\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.911459 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911215 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/343f8865-3015-4715-960c-a2e1418c6f7e-etc-tuned\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.911459 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911218 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-sys\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.911459 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911290 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/379bdc95-36d7-47f2-9e17-a96e51da6495-kubelet-config\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:03.911459 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911333 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-socket-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.911459 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911366 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-kubelet\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.911459 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911395 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-etc-selinux\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.911459 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911401 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-device-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.911845 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911497 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-device-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.911845 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911576 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-kubernetes\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.911845 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911613 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-cni-binary-copy\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.911845 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911711 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-node-log\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.911845 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911772 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.911845 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911808 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-slash\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.912146 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911859 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-socket-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.912146 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911875 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-run-netns\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.912146 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911904 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-etc-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.912146 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911870 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-slash\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.912146 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911966 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-node-log\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.912146 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911983 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-serviceca\") pod \"node-ca-rkg7g\" (UID: \"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01\") " pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:28:03.912146 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.911972 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.912146 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912036 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-cni-bin\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.912146 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912050 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-run-netns\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.912146 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912074 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gztg\" (UniqueName: \"kubernetes.io/projected/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-kube-api-access-5gztg\") pod \"node-ca-rkg7g\" (UID: \"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01\") " pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:28:03.912146 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912085 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.912146 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912105 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-cnibin\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.912146 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912106 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-cni-bin\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.912146 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912157 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-etc-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.912795 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912164 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-systemd-units\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.912795 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912194 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-cnibin\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.912795 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912208 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-var-lib-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.912795 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912234 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-systemd-units\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.912795 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912245 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjjl9\" (UniqueName: \"kubernetes.io/projected/82660df8-f278-4d7c-b02a-149527e30d27-kube-api-access-pjjl9\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.912795 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912280 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:03.912795 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912747 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-var-lib-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.913144 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912799 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-env-overrides\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.913144 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:03.912831 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:28:03.913144 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912897 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-sys-fs\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.913144 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:03.912927 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret podName:379bdc95-36d7-47f2-9e17-a96e51da6495 nodeName:}" failed. No retries permitted until 2026-11-04 05:28:04.412884932 +0000 UTC m=+2.100714559 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret") pod "global-pull-secret-syncer-69rl9" (UID: "379bdc95-36d7-47f2-9e17-a96e51da6495") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:28:03.913144 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.912835 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-sys-fs\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.913144 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913029 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-systemd\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.913144 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913042 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.913144 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913073 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-host\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.913144 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913105 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-serviceca\") pod \"node-ca-rkg7g\" (UID: \"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01\") " pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:28:03.913144 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913122 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/82660df8-f278-4d7c-b02a-149527e30d27-ovn-node-metrics-cert\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.913144 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913134 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-systemd\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913153 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-ovnkube-script-lib\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913189 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-host\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913234 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-env-overrides\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913281 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-var-lib-kubelet\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913322 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6rf9\" (UniqueName: \"kubernetes.io/projected/343f8865-3015-4715-960c-a2e1418c6f7e-kube-api-access-x6rf9\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913341 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913360 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913385 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysctl-d\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913405 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-var-lib-kubelet\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913410 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913448 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913466 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-log-socket\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913512 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-ovnkube-config\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913546 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-run-ovn-kubernetes\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913593 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913608 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.913665 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913619 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-systemd\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913652 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-run-ovn-kubernetes\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913663 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-run\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913689 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-registration-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913690 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysctl-d\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913704 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-systemd\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913734 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/379bdc95-36d7-47f2-9e17-a96e51da6495-dbus\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913567 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-log-socket\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913763 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-registration-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913751 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-ovnkube-script-lib\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913764 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-lib-modules\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913754 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-run\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913828 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvb55\" (UniqueName: \"kubernetes.io/projected/f4a51628-0aab-4614-a4ed-11f184521cd4-kube-api-access-nvb55\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913858 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc46j\" (UniqueName: \"kubernetes.io/projected/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-kube-api-access-rc46j\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913874 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/379bdc95-36d7-47f2-9e17-a96e51da6495-dbus\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913872 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-lib-modules\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913888 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-ovn\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.914446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913915 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-cni-netd\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.914999 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913938 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-modprobe-d\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.914999 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913938 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-ovn\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.914999 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913960 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysconfig\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.914999 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913979 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-cni-netd\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.914999 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.913982 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/343f8865-3015-4715-960c-a2e1418c6f7e-tmp\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.914999 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.914007 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-system-cni-dir\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.914999 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.914029 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysconfig\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.914999 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.914055 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-ovnkube-config\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.914999 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.914061 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-modprobe-d\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.914999 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.914076 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-system-cni-dir\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.914999 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.914874 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/343f8865-3015-4715-960c-a2e1418c6f7e-etc-tuned\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.915957 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.915936 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/82660df8-f278-4d7c-b02a-149527e30d27-ovn-node-metrics-cert\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.916041 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.916022 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/343f8865-3015-4715-960c-a2e1418c6f7e-tmp\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:03.922342 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.922321 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gztg\" (UniqueName: \"kubernetes.io/projected/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-kube-api-access-5gztg\") pod \"node-ca-rkg7g\" (UID: \"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01\") " pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:28:03.922626 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.922606 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjjl9\" (UniqueName: \"kubernetes.io/projected/82660df8-f278-4d7c-b02a-149527e30d27-kube-api-access-pjjl9\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:03.925799 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.925780 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvb55\" (UniqueName: \"kubernetes.io/projected/f4a51628-0aab-4614-a4ed-11f184521cd4-kube-api-access-nvb55\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:03.926007 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.925988 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc46j\" (UniqueName: \"kubernetes.io/projected/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-kube-api-access-rc46j\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:03.926594 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:03.926574 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6rf9\" (UniqueName: \"kubernetes.io/projected/343f8865-3015-4715-960c-a2e1418c6f7e-kube-api-access-x6rf9\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:04.016224 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.016181 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:28:04.035776 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.035743 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:28:04.059859 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.059832 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:28:04.071674 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.071652 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hk726" Nov 04 05:28:04.083838 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.083812 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:28:04.098942 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.098922 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:28:04.114397 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:04.114362 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7239836a_64ce_4847_91f4_9360cd652ea6.slice/crio-81d3b7d850ec27e11ee9ad7f1af5b6c3f78fd747b809dddf7099656a34dbed8f WatchSource:0}: Error finding container 81d3b7d850ec27e11ee9ad7f1af5b6c3f78fd747b809dddf7099656a34dbed8f: Status 404 returned error can't find the container with id 81d3b7d850ec27e11ee9ad7f1af5b6c3f78fd747b809dddf7099656a34dbed8f Nov 04 05:28:04.114691 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:04.114666 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58571c34b4dbebe904f2b5a148e16c40.slice/crio-cc61bb02f57f0ef6474fb326691082adb0a9cfd869f7df4548c847266c2dfa14 WatchSource:0}: Error finding container cc61bb02f57f0ef6474fb326691082adb0a9cfd869f7df4548c847266c2dfa14: Status 404 returned error can't find the container with id cc61bb02f57f0ef6474fb326691082adb0a9cfd869f7df4548c847266c2dfa14 Nov 04 05:28:04.114900 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:04.114881 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod343f8865_3015_4715_960c_a2e1418c6f7e.slice/crio-d6ffa17e77f3609e314bd8452f939603cc822d92360a8213c1bc926af7ff7e0c WatchSource:0}: Error finding container d6ffa17e77f3609e314bd8452f939603cc822d92360a8213c1bc926af7ff7e0c: Status 404 returned error can't find the container with id d6ffa17e77f3609e314bd8452f939603cc822d92360a8213c1bc926af7ff7e0c Nov 04 05:28:04.120681 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.120662 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:28:04.121098 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:04.121063 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bcebbdb_3bcd_4050_94b5_3139d6cf3d01.slice/crio-94135939bff49a0fa311a9f073dc3229f16aed854b2416ea53fd78eab0f2adb6 WatchSource:0}: Error finding container 94135939bff49a0fa311a9f073dc3229f16aed854b2416ea53fd78eab0f2adb6: Status 404 returned error can't find the container with id 94135939bff49a0fa311a9f073dc3229f16aed854b2416ea53fd78eab0f2adb6 Nov 04 05:28:04.122021 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:04.121989 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb1cdcff7ca15ea5212ef7ed52c2b8e3.slice/crio-88caa64b9200e78dbe39b56806babe65cda107d9b4711b1f4010c8c959b530b9 WatchSource:0}: Error finding container 88caa64b9200e78dbe39b56806babe65cda107d9b4711b1f4010c8c959b530b9: Status 404 returned error can't find the container with id 88caa64b9200e78dbe39b56806babe65cda107d9b4711b1f4010c8c959b530b9 Nov 04 05:28:04.127624 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.127603 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:28:04.137838 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:04.137801 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4a51628_0aab_4614_a4ed_11f184521cd4.slice/crio-90c9151a2fd867fa92e53aa4a1598e337ebe1bb24b81be305a6c9b7db4fa2aef WatchSource:0}: Error finding container 90c9151a2fd867fa92e53aa4a1598e337ebe1bb24b81be305a6c9b7db4fa2aef: Status 404 returned error can't find the container with id 90c9151a2fd867fa92e53aa4a1598e337ebe1bb24b81be305a6c9b7db4fa2aef Nov 04 05:28:04.144468 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.144436 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:04.151053 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.151036 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:28:04.153386 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:04.153365 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82660df8_f278_4d7c_b02a_149527e30d27.slice/crio-2d864e2af4c097fae3a6adc8a49ec7a6bf772854923fd38a9dd162cf84666664 WatchSource:0}: Error finding container 2d864e2af4c097fae3a6adc8a49ec7a6bf772854923fd38a9dd162cf84666664: Status 404 returned error can't find the container with id 2d864e2af4c097fae3a6adc8a49ec7a6bf772854923fd38a9dd162cf84666664 Nov 04 05:28:04.159304 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:04.159283 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbbe9ebe_9bf8_43fd_88ff_8b5b23126202.slice/crio-bd55eee618c292174656ff666436e101d1d4a8d706c27c8dfc320f56309eed53 WatchSource:0}: Error finding container bd55eee618c292174656ff666436e101d1d4a8d706c27c8dfc320f56309eed53: Status 404 returned error can't find the container with id bd55eee618c292174656ff666436e101d1d4a8d706c27c8dfc320f56309eed53 Nov 04 05:28:04.318464 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.318429 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:04.318659 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.318488 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4b8q\" (UniqueName: \"kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q\") pod \"network-check-target-mq5nl\" (UID: \"47be26f2-143e-4b49-824a-415e60f6140b\") " pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:04.318659 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:04.318597 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:28:04.318659 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:04.318622 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:28:04.318659 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:04.318635 2579 projected.go:196] Error preparing data for projected volume kube-api-access-h4b8q for pod openshift-network-diagnostics/network-check-target-mq5nl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:28:04.318841 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:04.318679 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q podName:47be26f2-143e-4b49-824a-415e60f6140b nodeName:}" failed. No retries permitted until 2026-11-04 05:28:05.318665292 +0000 UTC m=+3.006494901 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-h4b8q" (UniqueName: "kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q") pod "network-check-target-mq5nl" (UID: "47be26f2-143e-4b49-824a-415e60f6140b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:28:04.318841 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:04.318598 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:28:04.318841 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:04.318767 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs podName:efbcf64e-3ade-4343-a9c8-bf945b141a02 nodeName:}" failed. No retries permitted until 2026-11-04 05:28:05.318744724 +0000 UTC m=+3.006574335 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs") pod "network-metrics-daemon-brh9c" (UID: "efbcf64e-3ade-4343-a9c8-bf945b141a02") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:28:04.419280 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.419216 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:04.419456 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:04.419356 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:28:04.419456 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:04.419413 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret podName:379bdc95-36d7-47f2-9e17-a96e51da6495 nodeName:}" failed. No retries permitted until 2026-11-04 05:28:05.419400143 +0000 UTC m=+3.107229750 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret") pod "global-pull-secret-syncer-69rl9" (UID: "379bdc95-36d7-47f2-9e17-a96e51da6495") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:28:04.429576 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:04.429531 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80902534_76da_4e9d_8157_ce481e71f2f8.slice/crio-fce1cb0777caf725040593cf1a7448767e27710547736781c8687537530dd25d WatchSource:0}: Error finding container fce1cb0777caf725040593cf1a7448767e27710547736781c8687537530dd25d: Status 404 returned error can't find the container with id fce1cb0777caf725040593cf1a7448767e27710547736781c8687537530dd25d Nov 04 05:28:04.461877 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:04.461839 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7b4c728_a80c_403d_88d9_af4fb0835993.slice/crio-f5e1af278e75b5d52d7966e63de05e330aeb88b38c16ff21c0f46ccff5bf9732 WatchSource:0}: Error finding container f5e1af278e75b5d52d7966e63de05e330aeb88b38c16ff21c0f46ccff5bf9732: Status 404 returned error can't find the container with id f5e1af278e75b5d52d7966e63de05e330aeb88b38c16ff21c0f46ccff5bf9732 Nov 04 05:28:04.533591 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:04.533559 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod915dd778_3692_4f41_83d3_cc5d4e73d615.slice/crio-150dd6af1f9754f9d50d8336979b00a0ab8946403e61290e66a426bc4fcf64a4 WatchSource:0}: Error finding container 150dd6af1f9754f9d50d8336979b00a0ab8946403e61290e66a426bc4fcf64a4: Status 404 returned error can't find the container with id 150dd6af1f9754f9d50d8336979b00a0ab8946403e61290e66a426bc4fcf64a4 Nov 04 05:28:04.745021 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.744878 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 05:23:04 +0000 UTC" deadline="2028-05-17 04:59:34.56654454 +0000 UTC" Nov 04 05:28:04.745021 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.744986 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13439h31m29.821583608s" Nov 04 05:28:04.756318 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.756134 2579 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:28:04.858931 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.858896 2579 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:28:04.866383 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.865879 2579 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:28:04.980137 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.980099 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:04.980355 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:04.980250 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:04.980692 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:04.980670 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:04.980793 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:04.980771 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:05.008578 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.008474 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rkg7g" event={"ID":"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01","Type":"ContainerStarted","Data":"94135939bff49a0fa311a9f073dc3229f16aed854b2416ea53fd78eab0f2adb6"} Nov 04 05:28:05.014414 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.014207 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" event={"ID":"58571c34b4dbebe904f2b5a148e16c40","Type":"ContainerStarted","Data":"cc61bb02f57f0ef6474fb326691082adb0a9cfd869f7df4548c847266c2dfa14"} Nov 04 05:28:05.018539 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.018479 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hk726" event={"ID":"7239836a-64ce-4847-91f4-9360cd652ea6","Type":"ContainerStarted","Data":"81d3b7d850ec27e11ee9ad7f1af5b6c3f78fd747b809dddf7099656a34dbed8f"} Nov 04 05:28:05.026178 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.026153 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m66r7" event={"ID":"915dd778-3692-4f41-83d3-cc5d4e73d615","Type":"ContainerStarted","Data":"150dd6af1f9754f9d50d8336979b00a0ab8946403e61290e66a426bc4fcf64a4"} Nov 04 05:28:05.033563 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.033480 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m77gq" event={"ID":"f7b4c728-a80c-403d-88d9-af4fb0835993","Type":"ContainerStarted","Data":"f5e1af278e75b5d52d7966e63de05e330aeb88b38c16ff21c0f46ccff5bf9732"} Nov 04 05:28:05.047925 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.047851 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fvxf5" event={"ID":"80902534-76da-4e9d-8157-ce481e71f2f8","Type":"ContainerStarted","Data":"fce1cb0777caf725040593cf1a7448767e27710547736781c8687537530dd25d"} Nov 04 05:28:05.057854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.057823 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" event={"ID":"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202","Type":"ContainerStarted","Data":"bd55eee618c292174656ff666436e101d1d4a8d706c27c8dfc320f56309eed53"} Nov 04 05:28:05.072195 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.072078 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" event={"ID":"82660df8-f278-4d7c-b02a-149527e30d27","Type":"ContainerStarted","Data":"2d864e2af4c097fae3a6adc8a49ec7a6bf772854923fd38a9dd162cf84666664"} Nov 04 05:28:05.074688 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.074569 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z9djg" event={"ID":"f4a51628-0aab-4614-a4ed-11f184521cd4","Type":"ContainerStarted","Data":"90c9151a2fd867fa92e53aa4a1598e337ebe1bb24b81be305a6c9b7db4fa2aef"} Nov 04 05:28:05.084531 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.084466 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gbms4" event={"ID":"343f8865-3015-4715-960c-a2e1418c6f7e","Type":"ContainerStarted","Data":"d6ffa17e77f3609e314bd8452f939603cc822d92360a8213c1bc926af7ff7e0c"} Nov 04 05:28:05.092029 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.092000 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" event={"ID":"fb1cdcff7ca15ea5212ef7ed52c2b8e3","Type":"ContainerStarted","Data":"88caa64b9200e78dbe39b56806babe65cda107d9b4711b1f4010c8c959b530b9"} Nov 04 05:28:05.329178 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.329081 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:05.329383 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.329192 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4b8q\" (UniqueName: \"kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q\") pod \"network-check-target-mq5nl\" (UID: \"47be26f2-143e-4b49-824a-415e60f6140b\") " pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:05.329464 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:05.329399 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:28:05.329464 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:05.329425 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:28:05.329464 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:05.329441 2579 projected.go:196] Error preparing data for projected volume kube-api-access-h4b8q for pod openshift-network-diagnostics/network-check-target-mq5nl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:28:05.329673 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:05.329501 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q podName:47be26f2-143e-4b49-824a-415e60f6140b nodeName:}" failed. No retries permitted until 2026-11-04 05:28:07.329481153 +0000 UTC m=+5.017310765 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-h4b8q" (UniqueName: "kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q") pod "network-check-target-mq5nl" (UID: "47be26f2-143e-4b49-824a-415e60f6140b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:28:05.329975 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:05.329955 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:28:05.330053 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:05.330013 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs podName:efbcf64e-3ade-4343-a9c8-bf945b141a02 nodeName:}" failed. No retries permitted until 2026-11-04 05:28:07.329997313 +0000 UTC m=+5.017826927 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs") pod "network-metrics-daemon-brh9c" (UID: "efbcf64e-3ade-4343-a9c8-bf945b141a02") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:28:05.430625 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.430572 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:05.430852 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:05.430829 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:28:05.430945 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:05.430910 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret podName:379bdc95-36d7-47f2-9e17-a96e51da6495 nodeName:}" failed. No retries permitted until 2026-11-04 05:28:07.430889475 +0000 UTC m=+5.118719085 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret") pod "global-pull-secret-syncer-69rl9" (UID: "379bdc95-36d7-47f2-9e17-a96e51da6495") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:28:05.745885 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.745459 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 05:23:04 +0000 UTC" deadline="2028-06-30 21:33:28.661212691 +0000 UTC" Nov 04 05:28:05.745885 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.745498 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14512h5m22.915719983s" Nov 04 05:28:05.978104 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:05.978067 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:05.978312 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:05.978199 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:06.984411 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:06.983594 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:06.984411 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:06.983746 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:06.984411 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:06.984159 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:06.984411 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:06.984266 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:07.349284 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:07.349211 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:07.349460 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:07.349308 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4b8q\" (UniqueName: \"kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q\") pod \"network-check-target-mq5nl\" (UID: \"47be26f2-143e-4b49-824a-415e60f6140b\") " pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:07.349535 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:07.349467 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:28:07.349535 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:07.349486 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:28:07.349535 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:07.349500 2579 projected.go:196] Error preparing data for projected volume kube-api-access-h4b8q for pod openshift-network-diagnostics/network-check-target-mq5nl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:28:07.349698 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:07.349611 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q podName:47be26f2-143e-4b49-824a-415e60f6140b nodeName:}" failed. No retries permitted until 2026-11-04 05:28:11.349570782 +0000 UTC m=+9.037400412 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-h4b8q" (UniqueName: "kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q") pod "network-check-target-mq5nl" (UID: "47be26f2-143e-4b49-824a-415e60f6140b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:28:07.349990 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:07.349890 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:28:07.349990 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:07.349962 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs podName:efbcf64e-3ade-4343-a9c8-bf945b141a02 nodeName:}" failed. No retries permitted until 2026-11-04 05:28:11.349943571 +0000 UTC m=+9.037773180 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs") pod "network-metrics-daemon-brh9c" (UID: "efbcf64e-3ade-4343-a9c8-bf945b141a02") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:28:07.451377 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:07.449949 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:07.451377 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:07.450078 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:28:07.451377 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:07.450122 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret podName:379bdc95-36d7-47f2-9e17-a96e51da6495 nodeName:}" failed. No retries permitted until 2026-11-04 05:28:11.450110339 +0000 UTC m=+9.137939947 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret") pod "global-pull-secret-syncer-69rl9" (UID: "379bdc95-36d7-47f2-9e17-a96e51da6495") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:28:07.977866 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:07.977753 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:07.978062 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:07.977900 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:08.977897 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:08.977587 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:08.978695 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:08.977652 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:08.978695 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:08.978002 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:08.978695 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:08.978089 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:09.132254 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:09.132205 2579 generic.go:356] "Generic (PLEG): container finished" podID="f4a51628-0aab-4614-a4ed-11f184521cd4" containerID="fbf7189995ced8b432ac08e3fe4c38b967eda51c6ec4a3297345a7f826200502" exitCode=0 Nov 04 05:28:09.132406 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:09.132324 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z9djg" event={"ID":"f4a51628-0aab-4614-a4ed-11f184521cd4","Type":"ContainerDied","Data":"fbf7189995ced8b432ac08e3fe4c38b967eda51c6ec4a3297345a7f826200502"} Nov 04 05:28:09.162156 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:09.161700 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" event={"ID":"fb1cdcff7ca15ea5212ef7ed52c2b8e3","Type":"ContainerStarted","Data":"9802c542b62766d90de88595528428a4ebb10e446916a80e13f5815033bccc9b"} Nov 04 05:28:09.173498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:09.172856 2579 generic.go:356] "Generic (PLEG): container finished" podID="58571c34b4dbebe904f2b5a148e16c40" containerID="89f7e054c5a795fefa73216b0248ffbf88d728ca35f5dae8f5f9d551b556b29b" exitCode=0 Nov 04 05:28:09.173498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:09.172898 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" event={"ID":"58571c34b4dbebe904f2b5a148e16c40","Type":"ContainerDied","Data":"89f7e054c5a795fefa73216b0248ffbf88d728ca35f5dae8f5f9d551b556b29b"} Nov 04 05:28:09.177330 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:09.177259 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" podStartSLOduration=6.177241014 podStartE2EDuration="6.177241014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:28:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:09.176892728 +0000 UTC m=+6.864722360" watchObservedRunningTime="2026-11-04 05:28:09.177241014 +0000 UTC m=+6.865070648" Nov 04 05:28:09.977855 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:09.977419 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:09.977855 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:09.977549 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:10.178537 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:10.178507 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" event={"ID":"58571c34b4dbebe904f2b5a148e16c40","Type":"ContainerStarted","Data":"af400f861c41f3a1bfc6a2f1fd869d39df5a49f001297922376d2384881275e5"} Nov 04 05:28:10.183704 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:10.183682 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m66r7" event={"ID":"915dd778-3692-4f41-83d3-cc5d4e73d615","Type":"ContainerStarted","Data":"0f1e58bd2e8531e0aeb78bba5d4e5fb552a8108cacedf230ddd3567ebc839a09"} Nov 04 05:28:10.220855 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:10.220803 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" podStartSLOduration=7.220785454 podStartE2EDuration="7.220785454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:28:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:10.193484068 +0000 UTC m=+7.881313700" watchObservedRunningTime="2026-11-04 05:28:10.220785454 +0000 UTC m=+7.908615088" Nov 04 05:28:10.221043 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:10.220999 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-m66r7" podStartSLOduration=2.678803427 podStartE2EDuration="7.22099133s" totalImagesPullingTime="4.542187903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:10.220596034 +0000 UTC m=+7.908425662" watchObservedRunningTime="2026-11-04 05:28:10.22099133 +0000 UTC m=+7.908820960" Nov 04 05:28:10.979619 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:10.979588 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:10.980066 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:10.979710 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:10.980112 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:10.980082 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:10.980206 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:10.980185 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:11.188110 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:11.188028 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" event={"ID":"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202","Type":"ContainerStarted","Data":"6418075669c2d323feaa03636a82ede7a02e5a418891a39ee25ff78f4cc9291a"} Nov 04 05:28:11.192717 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:11.192289 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rkg7g" event={"ID":"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01","Type":"ContainerStarted","Data":"6120bcaae076565c105c2203a61c31b2acb1c634f8ae28a9da5931b3dda9e652"} Nov 04 05:28:11.206783 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:11.206725 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rkg7g" podStartSLOduration=2.125626934 podStartE2EDuration="8.206710576s" totalImagesPullingTime="6.081083642s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:11.205347712 +0000 UTC m=+8.893177350" watchObservedRunningTime="2026-11-04 05:28:11.206710576 +0000 UTC m=+8.894540208" Nov 04 05:28:11.384551 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:11.384516 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4b8q\" (UniqueName: \"kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q\") pod \"network-check-target-mq5nl\" (UID: \"47be26f2-143e-4b49-824a-415e60f6140b\") " pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:11.384689 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:11.384668 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:11.384750 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:11.384707 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:28:11.384750 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:11.384733 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:28:11.384750 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:11.384747 2579 projected.go:196] Error preparing data for projected volume kube-api-access-h4b8q for pod openshift-network-diagnostics/network-check-target-mq5nl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:28:11.384887 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:11.384778 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:28:11.384887 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:11.384812 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q podName:47be26f2-143e-4b49-824a-415e60f6140b nodeName:}" failed. No retries permitted until 2026-11-04 05:28:19.384794995 +0000 UTC m=+17.072624607 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-h4b8q" (UniqueName: "kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q") pod "network-check-target-mq5nl" (UID: "47be26f2-143e-4b49-824a-415e60f6140b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:28:11.384887 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:11.384833 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs podName:efbcf64e-3ade-4343-a9c8-bf945b141a02 nodeName:}" failed. No retries permitted until 2026-11-04 05:28:19.384823714 +0000 UTC m=+17.072653321 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs") pod "network-metrics-daemon-brh9c" (UID: "efbcf64e-3ade-4343-a9c8-bf945b141a02") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:28:11.486096 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:11.486015 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:11.486296 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:11.486201 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:28:11.486296 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:11.486286 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret podName:379bdc95-36d7-47f2-9e17-a96e51da6495 nodeName:}" failed. No retries permitted until 2026-11-04 05:28:19.486268505 +0000 UTC m=+17.174098115 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret") pod "global-pull-secret-syncer-69rl9" (UID: "379bdc95-36d7-47f2-9e17-a96e51da6495") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:28:11.977287 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:11.977191 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:11.977436 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:11.977383 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:12.665716 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:12.665684 2579 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 05:28:12.822763 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:12.822623 2579 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T05:28:12.665709381Z","UUID":"b6ba15aa-7c48-4c9e-9003-12a76954b56e","Handler":null,"Name":"","Endpoint":""} Nov 04 05:28:12.829386 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:12.829361 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 05:28:12.829526 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:12.829407 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 05:28:12.979051 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:12.977354 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:12.979051 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:12.977499 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:12.979051 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:12.977894 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:12.979051 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:12.978008 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:13.199962 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:13.199889 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" event={"ID":"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202","Type":"ContainerStarted","Data":"9ff1663075c95ddbcca705e7b28dd43147f6a24f0cd47d49b862534635028a27"} Nov 04 05:28:13.202875 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:13.202829 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hk726" event={"ID":"7239836a-64ce-4847-91f4-9360cd652ea6","Type":"ContainerStarted","Data":"6e73821c205f6d79f8b90ffa32ed22e36f13be2dc4276f6bf308efbf907859f9"} Nov 04 05:28:13.236018 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:13.235951 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-hk726" podStartSLOduration=1.487538574 podStartE2EDuration="10.235932676s" totalImagesPullingTime="8.748394102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:13.218523868 +0000 UTC m=+10.906353496" watchObservedRunningTime="2026-11-04 05:28:13.235932676 +0000 UTC m=+10.923762307" Nov 04 05:28:13.977429 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:13.977391 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:13.977848 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:13.977533 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:14.207057 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:14.206826 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fvxf5" event={"ID":"80902534-76da-4e9d-8157-ce481e71f2f8","Type":"ContainerStarted","Data":"b8d61c383f0cacbc9d2282edf79242e48ea4240f27d80a5a990028a38d138ff4"} Nov 04 05:28:14.209998 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:14.209968 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" event={"ID":"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202","Type":"ContainerStarted","Data":"d2cdbe6053694c7771935e69116dcd63f1516be1ebc8a97d1a976e935b37424d"} Nov 04 05:28:14.212028 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:14.212000 2579 generic.go:356] "Generic (PLEG): container finished" podID="f4a51628-0aab-4614-a4ed-11f184521cd4" containerID="a3698660193295b886168c0a699a62446521b18a6d4b8f3f8c11a2e4fcdcbecf" exitCode=0 Nov 04 05:28:14.212966 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:14.212940 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z9djg" event={"ID":"f4a51628-0aab-4614-a4ed-11f184521cd4","Type":"ContainerDied","Data":"a3698660193295b886168c0a699a62446521b18a6d4b8f3f8c11a2e4fcdcbecf"} Nov 04 05:28:14.220331 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:14.220279 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-fvxf5" podStartSLOduration=2.399058281 podStartE2EDuration="11.220262791s" totalImagesPullingTime="8.82120451s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:14.219753431 +0000 UTC m=+11.907583063" watchObservedRunningTime="2026-11-04 05:28:14.220262791 +0000 UTC m=+11.908092421" Nov 04 05:28:14.256757 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:14.256708 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" podStartSLOduration=2.064541327 podStartE2EDuration="11.256693882s" totalImagesPullingTime="9.192152555s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:28:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:14.256052842 +0000 UTC m=+11.943882471" watchObservedRunningTime="2026-11-04 05:28:14.256693882 +0000 UTC m=+11.944523511" Nov 04 05:28:14.793025 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:14.792799 2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:28:14.793494 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:14.793471 2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:28:14.981625 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:14.976896 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:14.981625 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:14.977032 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:14.981625 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:14.977260 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:14.981625 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:14.977379 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:15.216363 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:15.216326 2579 generic.go:356] "Generic (PLEG): container finished" podID="f4a51628-0aab-4614-a4ed-11f184521cd4" containerID="79313f40b83df9ed661abc441aade08d8646e7c2e037298d783a951e6cde2ffa" exitCode=0 Nov 04 05:28:15.216482 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:15.216412 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z9djg" event={"ID":"f4a51628-0aab-4614-a4ed-11f184521cd4","Type":"ContainerDied","Data":"79313f40b83df9ed661abc441aade08d8646e7c2e037298d783a951e6cde2ffa"} Nov 04 05:28:15.217998 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:15.217964 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gbms4" event={"ID":"343f8865-3015-4715-960c-a2e1418c6f7e","Type":"ContainerStarted","Data":"f60374762c7a6e924cb01b177e224973d5ab2aa71e5f5d0fa8bae0fda59eb010"} Nov 04 05:28:15.219551 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:15.219532 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m77gq" event={"ID":"f7b4c728-a80c-403d-88d9-af4fb0835993","Type":"ContainerStarted","Data":"e3ae727326b32294e5f43da17d167d643363cc15975e9920e86882581b50125b"} Nov 04 05:28:15.246942 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:15.246901 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m77gq" podStartSLOduration=3.308236495 podStartE2EDuration="12.246887521s" totalImagesPullingTime="8.938651026s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:15.246618112 +0000 UTC m=+12.934447741" watchObservedRunningTime="2026-11-04 05:28:15.246887521 +0000 UTC m=+12.934717149" Nov 04 05:28:15.976856 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:15.976830 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:15.976984 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:15.976933 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:16.224633 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:16.224548 2579 generic.go:356] "Generic (PLEG): container finished" podID="f4a51628-0aab-4614-a4ed-11f184521cd4" containerID="00d8a6c2e92669146e386a88bca65888b34ea3b007480f849f2d44509b5224c5" exitCode=0 Nov 04 05:28:16.225145 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:16.224632 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z9djg" event={"ID":"f4a51628-0aab-4614-a4ed-11f184521cd4","Type":"ContainerDied","Data":"00d8a6c2e92669146e386a88bca65888b34ea3b007480f849f2d44509b5224c5"} Nov 04 05:28:16.258979 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:16.258924 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gbms4" podStartSLOduration=2.26843854 podStartE2EDuration="13.258905696s" totalImagesPullingTime="10.990467156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:16.258531842 +0000 UTC m=+13.946361483" watchObservedRunningTime="2026-11-04 05:28:16.258905696 +0000 UTC m=+13.946735327" Nov 04 05:28:16.979861 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:16.979828 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:16.980038 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:16.979944 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:16.980038 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:16.980018 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:16.980168 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:16.980090 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:17.977335 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:17.977301 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:17.977925 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:17.977409 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:18.977022 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:18.976841 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:18.977201 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:18.976884 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:18.977201 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:18.977128 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:18.977201 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:18.977177 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:19.233444 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:19.233378 2579 generic.go:356] "Generic (PLEG): container finished" podID="f4a51628-0aab-4614-a4ed-11f184521cd4" containerID="c9cc9d148cc1991401cc7deb85c90ff87bbb4c12c2747bcaf3f7b3792781c11c" exitCode=0 Nov 04 05:28:19.233444 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:19.233415 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z9djg" event={"ID":"f4a51628-0aab-4614-a4ed-11f184521cd4","Type":"ContainerDied","Data":"c9cc9d148cc1991401cc7deb85c90ff87bbb4c12c2747bcaf3f7b3792781c11c"} Nov 04 05:28:19.449040 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:19.449007 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4b8q\" (UniqueName: \"kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q\") pod \"network-check-target-mq5nl\" (UID: \"47be26f2-143e-4b49-824a-415e60f6140b\") " pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:19.449201 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:19.449067 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:19.449201 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:19.449188 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:28:19.449337 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:19.449200 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:28:19.449337 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:19.449239 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:28:19.449337 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:19.449253 2579 projected.go:196] Error preparing data for projected volume kube-api-access-h4b8q for pod openshift-network-diagnostics/network-check-target-mq5nl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:28:19.449337 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:19.449274 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs podName:efbcf64e-3ade-4343-a9c8-bf945b141a02 nodeName:}" failed. No retries permitted until 2026-11-04 05:28:35.449256104 +0000 UTC m=+33.137085726 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs") pod "network-metrics-daemon-brh9c" (UID: "efbcf64e-3ade-4343-a9c8-bf945b141a02") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:28:19.449337 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:19.449298 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q podName:47be26f2-143e-4b49-824a-415e60f6140b nodeName:}" failed. No retries permitted until 2026-11-04 05:28:35.449283617 +0000 UTC m=+33.137113223 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-h4b8q" (UniqueName: "kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q") pod "network-check-target-mq5nl" (UID: "47be26f2-143e-4b49-824a-415e60f6140b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:28:19.549703 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:19.549678 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:19.549808 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:19.549795 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:28:19.549858 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:19.549849 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret podName:379bdc95-36d7-47f2-9e17-a96e51da6495 nodeName:}" failed. No retries permitted until 2026-11-04 05:28:35.549838452 +0000 UTC m=+33.237668058 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret") pod "global-pull-secret-syncer-69rl9" (UID: "379bdc95-36d7-47f2-9e17-a96e51da6495") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:28:19.977756 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:19.977726 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:19.977902 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:19.977848 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:20.237798 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:20.237594 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-slrx2_82660df8-f278-4d7c-b02a-149527e30d27/ovn-acl-logging/0.log" Nov 04 05:28:20.238191 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:20.238044 2579 generic.go:356] "Generic (PLEG): container finished" podID="82660df8-f278-4d7c-b02a-149527e30d27" containerID="0caaa4108d97a74bdc828247da5f268f726e7bb7e92925f3e1c1202384f01e02" exitCode=1 Nov 04 05:28:20.238191 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:20.238119 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" event={"ID":"82660df8-f278-4d7c-b02a-149527e30d27","Type":"ContainerStarted","Data":"3f92528a423ce7b7b44379a807273b45b40b2348a3fe21b169ea7f551e52e3ec"} Nov 04 05:28:20.238302 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:20.238209 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" event={"ID":"82660df8-f278-4d7c-b02a-149527e30d27","Type":"ContainerStarted","Data":"cb36528291ede6149fd0fc6599e75866f08792fd7f6a4685691820a7c59bb415"} Nov 04 05:28:20.238302 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:20.238269 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" event={"ID":"82660df8-f278-4d7c-b02a-149527e30d27","Type":"ContainerStarted","Data":"92d24e89266fdadfb68010c5cebf9a82d7cebf1fd2e3b1224bb3cf5cd3e9a3e9"} Nov 04 05:28:20.238302 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:20.238297 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" event={"ID":"82660df8-f278-4d7c-b02a-149527e30d27","Type":"ContainerStarted","Data":"9de851bd38decfb67ece49bd36705f8624c939a722dc74d3921575ffd725de26"} Nov 04 05:28:20.238405 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:20.238341 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" event={"ID":"82660df8-f278-4d7c-b02a-149527e30d27","Type":"ContainerDied","Data":"0caaa4108d97a74bdc828247da5f268f726e7bb7e92925f3e1c1202384f01e02"} Nov 04 05:28:20.238405 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:20.238387 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" event={"ID":"82660df8-f278-4d7c-b02a-149527e30d27","Type":"ContainerStarted","Data":"a463477b6612dd27cbf87d98c99c9c9f070910d6b68dbfbca366d134981dc918"} Nov 04 05:28:20.241118 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:20.241093 2579 generic.go:356] "Generic (PLEG): container finished" podID="f4a51628-0aab-4614-a4ed-11f184521cd4" containerID="1a45106a3e204439c36c8d181bebb9116c9b2b5b0c33c91759de2e08c4adae11" exitCode=0 Nov 04 05:28:20.241236 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:20.241133 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z9djg" event={"ID":"f4a51628-0aab-4614-a4ed-11f184521cd4","Type":"ContainerDied","Data":"1a45106a3e204439c36c8d181bebb9116c9b2b5b0c33c91759de2e08c4adae11"} Nov 04 05:28:20.977332 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:20.977301 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:20.977517 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:20.977407 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:20.977517 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:20.977440 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:20.977517 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:20.977508 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:21.246165 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:21.246089 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z9djg" event={"ID":"f4a51628-0aab-4614-a4ed-11f184521cd4","Type":"ContainerStarted","Data":"f873b57cb33ee1843f3a6f7012cb4cc312d7dc16b95a131702928b4c66aaed88"} Nov 04 05:28:21.267943 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:21.267904 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-z9djg" podStartSLOduration=7.647499631 podStartE2EDuration="18.267891368s" totalImagesPullingTime="10.620391737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:28:03 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:21.265757627 +0000 UTC m=+18.953587255" watchObservedRunningTime="2026-11-04 05:28:21.267891368 +0000 UTC m=+18.955720995" Nov 04 05:28:21.977719 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:21.977694 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:21.977836 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:21.977818 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:22.251519 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:22.251447 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-slrx2_82660df8-f278-4d7c-b02a-149527e30d27/ovn-acl-logging/0.log" Nov 04 05:28:22.251989 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:22.251845 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" event={"ID":"82660df8-f278-4d7c-b02a-149527e30d27","Type":"ContainerStarted","Data":"2d6bc0f8a78ef26ce6925c0755d9092ac6fa976ef01410aca56c1d50d74cbd9d"} Nov 04 05:28:22.977895 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:22.977865 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:22.978035 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:22.978004 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:22.978090 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:22.978056 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:22.978174 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:22.978156 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:23.746849 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:23.746829 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fvxf5_80902534-76da-4e9d-8157-ce481e71f2f8/dns-node-resolver/0.log" Nov 04 05:28:23.977531 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:23.977494 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:23.977710 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:23.977590 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:24.260020 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:24.259948 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-slrx2_82660df8-f278-4d7c-b02a-149527e30d27/ovn-acl-logging/0.log" Nov 04 05:28:24.260348 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:24.260324 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" event={"ID":"82660df8-f278-4d7c-b02a-149527e30d27","Type":"ContainerStarted","Data":"5e92a266a568b62684654ec12f3745169f26fb28a9734fe31ca7a36a6772820f"} Nov 04 05:28:24.263151 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:24.261266 2579 scope.go:122] "RemoveContainer" containerID="0caaa4108d97a74bdc828247da5f268f726e7bb7e92925f3e1c1202384f01e02" Nov 04 05:28:24.263151 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:24.262259 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:24.263151 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:24.262374 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:24.263151 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:24.262438 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:24.278628 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:24.278606 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:24.279835 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:24.279820 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:24.930353 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:24.930326 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rkg7g_4bcebbdb-3bcd-4050-94b5-3139d6cf3d01/node-ca/0.log" Nov 04 05:28:24.977611 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:24.977584 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:24.977759 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:24.977684 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:24.977759 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:24.977732 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:24.977852 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:24.977788 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:25.266152 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:25.266085 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-slrx2_82660df8-f278-4d7c-b02a-149527e30d27/ovn-acl-logging/0.log" Nov 04 05:28:25.266450 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:25.266430 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" event={"ID":"82660df8-f278-4d7c-b02a-149527e30d27","Type":"ContainerStarted","Data":"3b0d32f26ba6b38d45f65efb8d606defc20885b4dfcfb3b4809b8d50bfcb30f9"} Nov 04 05:28:25.292482 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:25.292440 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" podStartSLOduration=6.941651877 podStartE2EDuration="22.292426703s" totalImagesPullingTime="15.350774826s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:25.290716807 +0000 UTC m=+22.978546434" watchObservedRunningTime="2026-11-04 05:28:25.292426703 +0000 UTC m=+22.980256330" Nov 04 05:28:25.976963 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:25.976935 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:25.977323 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:25.977025 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:26.977826 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:26.977584 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:26.978279 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:26.977660 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:26.978279 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:26.977936 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:26.978279 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:26.977981 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:27.977534 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:27.977505 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:27.977720 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:27.977604 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:28.977438 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:28.977410 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:28.977848 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:28.977519 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:28.977848 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:28.977600 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:28.977848 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:28.977703 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:29.977003 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:29.976973 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:29.977242 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:29.977070 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:30.976816 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:30.976778 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:30.977322 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:30.976779 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:30.977322 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:30.976887 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:30.977322 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:30.976935 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:31.977437 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:31.977402 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:31.977809 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:31.977502 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:32.978203 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:32.978171 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:32.978601 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:32.978271 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:32.978601 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:32.978367 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:32.978601 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:32.978455 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:33.933113 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:33.933080 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:28:33.933330 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:33.933213 2579 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 05:28:33.934041 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:33.933945 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:28:33.977446 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:33.977418 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:33.977566 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:33.977501 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:34.977563 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:34.977528 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:34.977979 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:34.977560 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:34.977979 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:34.977636 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:34.977979 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:34.977755 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:35.462063 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:35.462034 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4b8q\" (UniqueName: \"kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q\") pod \"network-check-target-mq5nl\" (UID: \"47be26f2-143e-4b49-824a-415e60f6140b\") " pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:35.462247 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:35.462077 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:35.462247 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:35.462176 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:28:35.462247 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:35.462200 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:28:35.462247 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:35.462218 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:28:35.462247 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:35.462241 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs podName:efbcf64e-3ade-4343-a9c8-bf945b141a02 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:07.462212366 +0000 UTC m=+65.150041985 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs") pod "network-metrics-daemon-brh9c" (UID: "efbcf64e-3ade-4343-a9c8-bf945b141a02") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:28:35.462247 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:35.462242 2579 projected.go:196] Error preparing data for projected volume kube-api-access-h4b8q for pod openshift-network-diagnostics/network-check-target-mq5nl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:28:35.462446 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:35.462280 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q podName:47be26f2-143e-4b49-824a-415e60f6140b nodeName:}" failed. No retries permitted until 2026-11-04 05:29:07.462273423 +0000 UTC m=+65.150103029 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-h4b8q" (UniqueName: "kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q") pod "network-check-target-mq5nl" (UID: "47be26f2-143e-4b49-824a-415e60f6140b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:28:35.562928 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:35.562894 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:35.563091 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:35.563042 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:28:35.563129 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:35.563100 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret podName:379bdc95-36d7-47f2-9e17-a96e51da6495 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:07.563086933 +0000 UTC m=+65.250916539 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret") pod "global-pull-secret-syncer-69rl9" (UID: "379bdc95-36d7-47f2-9e17-a96e51da6495") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:28:35.977009 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:35.976979 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:35.977193 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:35.977089 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:36.977571 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:36.977537 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:36.978069 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:36.977672 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:36.978069 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:36.977714 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:36.978069 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:36.977783 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:37.977622 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:37.977593 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:37.978067 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:37.977686 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:38.977300 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:38.977267 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:38.977492 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:38.977371 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:38.977492 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:38.977428 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:38.977605 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:38.977520 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:39.977545 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:39.977522 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:39.977921 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:39.977608 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:40.977251 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:40.977191 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:40.977423 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:40.977326 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:40.977423 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:40.977380 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:40.977510 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:40.977492 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:41.870497 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:41.870423 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-69rl9"] Nov 04 05:28:41.871076 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:41.871051 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:41.871236 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:41.871202 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:41.876584 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:41.876280 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mq5nl"] Nov 04 05:28:41.876584 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:41.876350 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-brh9c"] Nov 04 05:28:41.876584 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:41.876476 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:41.876813 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:41.876632 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:41.876813 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:41.876680 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:41.876813 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:41.876803 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:43.977079 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:43.977049 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:43.977079 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:43.977077 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:43.977510 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:43.977140 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:43.977510 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:43.977183 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:43.977510 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:43.977291 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:43.977510 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:43.977330 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:45.977604 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:45.977413 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:45.977979 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:45.977465 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:45.977979 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:45.977676 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mq5nl" podUID="47be26f2-143e-4b49-824a-415e60f6140b" Nov 04 05:28:45.977979 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:45.977478 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:45.977979 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:45.977746 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69rl9" podUID="379bdc95-36d7-47f2-9e17-a96e51da6495" Nov 04 05:28:45.977979 ip-10-0-128-126 kubenswrapper[2579]: E1104 05:28:45.977813 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brh9c" podUID="efbcf64e-3ade-4343-a9c8-bf945b141a02" Nov 04 05:28:46.572251 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.572212 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-126.us-east-2.compute.internal" event="NodeReady" Nov 04 05:28:46.572458 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.572379 2579 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Nov 04 05:28:46.620248 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.619552 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-skkvk"] Nov 04 05:28:46.621679 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.621657 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2k9r5"] Nov 04 05:28:46.621805 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.621789 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.623186 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.623166 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2k9r5" Nov 04 05:28:46.625371 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.625356 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 05:28:46.625928 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.625908 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ghvrd\"" Nov 04 05:28:46.626306 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.626288 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 05:28:46.626515 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.626499 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 05:28:46.627049 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.627026 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 05:28:46.627049 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.627032 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 05:28:46.627168 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.627083 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 05:28:46.627168 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.627084 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-gqfxb\"" Nov 04 05:28:46.627168 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.627086 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 05:28:46.633527 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.633505 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2k9r5"] Nov 04 05:28:46.638516 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.637023 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-skkvk"] Nov 04 05:28:46.713693 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.713663 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9zfz8"] Nov 04 05:28:46.715506 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.715492 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9zfz8" Nov 04 05:28:46.726904 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.726876 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 05:28:46.727036 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.726991 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qm9rq\"" Nov 04 05:28:46.727103 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.727057 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 05:28:46.728321 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.728288 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9zfz8"] Nov 04 05:28:46.744988 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.744965 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c93278df-81cb-480e-9151-ba3056f43d74-crio-socket\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.745090 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.744990 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c93278df-81cb-480e-9151-ba3056f43d74-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.745090 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.745013 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt7bc\" (UniqueName: \"kubernetes.io/projected/9f075074-0903-4de4-b247-59a26f5b5f95-kube-api-access-mt7bc\") pod \"ingress-canary-2k9r5\" (UID: \"9f075074-0903-4de4-b247-59a26f5b5f95\") " pod="openshift-ingress-canary/ingress-canary-2k9r5" Nov 04 05:28:46.745156 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.745126 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f075074-0903-4de4-b247-59a26f5b5f95-cert\") pod \"ingress-canary-2k9r5\" (UID: \"9f075074-0903-4de4-b247-59a26f5b5f95\") " pod="openshift-ingress-canary/ingress-canary-2k9r5" Nov 04 05:28:46.745188 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.745159 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c93278df-81cb-480e-9151-ba3056f43d74-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.745188 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.745178 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c93278df-81cb-480e-9151-ba3056f43d74-data-volume\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.745299 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.745199 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzdc9\" (UniqueName: \"kubernetes.io/projected/c93278df-81cb-480e-9151-ba3056f43d74-kube-api-access-wzdc9\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.845792 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.845713 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt7bc\" (UniqueName: \"kubernetes.io/projected/9f075074-0903-4de4-b247-59a26f5b5f95-kube-api-access-mt7bc\") pod \"ingress-canary-2k9r5\" (UID: \"9f075074-0903-4de4-b247-59a26f5b5f95\") " pod="openshift-ingress-canary/ingress-canary-2k9r5" Nov 04 05:28:46.845792 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.845749 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e19ce810-1d70-46db-a8ce-d824a9ffc81e-config-volume\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:28:46.845792 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.845772 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e19ce810-1d70-46db-a8ce-d824a9ffc81e-metrics-tls\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:28:46.846057 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.845921 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8wjt\" (UniqueName: \"kubernetes.io/projected/e19ce810-1d70-46db-a8ce-d824a9ffc81e-kube-api-access-x8wjt\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:28:46.846057 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.845966 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f075074-0903-4de4-b247-59a26f5b5f95-cert\") pod \"ingress-canary-2k9r5\" (UID: \"9f075074-0903-4de4-b247-59a26f5b5f95\") " pod="openshift-ingress-canary/ingress-canary-2k9r5" Nov 04 05:28:46.846057 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.846041 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e19ce810-1d70-46db-a8ce-d824a9ffc81e-tmp-dir\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:28:46.846197 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.846076 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c93278df-81cb-480e-9151-ba3056f43d74-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.846197 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.846115 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c93278df-81cb-480e-9151-ba3056f43d74-data-volume\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.846197 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.846141 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzdc9\" (UniqueName: \"kubernetes.io/projected/c93278df-81cb-480e-9151-ba3056f43d74-kube-api-access-wzdc9\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.846380 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.846293 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c93278df-81cb-480e-9151-ba3056f43d74-crio-socket\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.846380 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.846324 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c93278df-81cb-480e-9151-ba3056f43d74-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.846502 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.846481 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c93278df-81cb-480e-9151-ba3056f43d74-crio-socket\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.846568 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.846550 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c93278df-81cb-480e-9151-ba3056f43d74-data-volume\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.846735 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.846715 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c93278df-81cb-480e-9151-ba3056f43d74-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.849814 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.849781 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c93278df-81cb-480e-9151-ba3056f43d74-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.849928 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.849895 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f075074-0903-4de4-b247-59a26f5b5f95-cert\") pod \"ingress-canary-2k9r5\" (UID: \"9f075074-0903-4de4-b247-59a26f5b5f95\") " pod="openshift-ingress-canary/ingress-canary-2k9r5" Nov 04 05:28:46.853794 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.853758 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzdc9\" (UniqueName: \"kubernetes.io/projected/c93278df-81cb-480e-9151-ba3056f43d74-kube-api-access-wzdc9\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.853954 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.853825 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt7bc\" (UniqueName: \"kubernetes.io/projected/9f075074-0903-4de4-b247-59a26f5b5f95-kube-api-access-mt7bc\") pod \"ingress-canary-2k9r5\" (UID: \"9f075074-0903-4de4-b247-59a26f5b5f95\") " pod="openshift-ingress-canary/ingress-canary-2k9r5" Nov 04 05:28:46.940763 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.940737 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:28:46.944865 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.944848 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2k9r5" Nov 04 05:28:46.947565 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.947546 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e19ce810-1d70-46db-a8ce-d824a9ffc81e-config-volume\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:28:46.947650 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.947573 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e19ce810-1d70-46db-a8ce-d824a9ffc81e-metrics-tls\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:28:46.947650 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.947604 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8wjt\" (UniqueName: \"kubernetes.io/projected/e19ce810-1d70-46db-a8ce-d824a9ffc81e-kube-api-access-x8wjt\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:28:46.947650 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.947628 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e19ce810-1d70-46db-a8ce-d824a9ffc81e-tmp-dir\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:28:46.948011 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.947994 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e19ce810-1d70-46db-a8ce-d824a9ffc81e-tmp-dir\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:28:46.948277 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.948258 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e19ce810-1d70-46db-a8ce-d824a9ffc81e-config-volume\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:28:46.950018 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.949995 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e19ce810-1d70-46db-a8ce-d824a9ffc81e-metrics-tls\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:28:46.955492 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:46.955469 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8wjt\" (UniqueName: \"kubernetes.io/projected/e19ce810-1d70-46db-a8ce-d824a9ffc81e-kube-api-access-x8wjt\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:28:47.025584 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.025550 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9zfz8" Nov 04 05:28:47.232088 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.232010 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2k9r5"] Nov 04 05:28:47.235048 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.235024 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-skkvk"] Nov 04 05:28:47.235472 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:47.235447 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f075074_0903_4de4_b247_59a26f5b5f95.slice/crio-ffc140d9d862cd089d5b3cd81b402ab2707df60ab6400c5a38fef3123c5b7074 WatchSource:0}: Error finding container ffc140d9d862cd089d5b3cd81b402ab2707df60ab6400c5a38fef3123c5b7074: Status 404 returned error can't find the container with id ffc140d9d862cd089d5b3cd81b402ab2707df60ab6400c5a38fef3123c5b7074 Nov 04 05:28:47.237479 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:47.237450 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc93278df_81cb_480e_9151_ba3056f43d74.slice/crio-37daf502ec6a620564fe64fcf8e755709761e5b543b611a8c11c693e07680569 WatchSource:0}: Error finding container 37daf502ec6a620564fe64fcf8e755709761e5b543b611a8c11c693e07680569: Status 404 returned error can't find the container with id 37daf502ec6a620564fe64fcf8e755709761e5b543b611a8c11c693e07680569 Nov 04 05:28:47.294752 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.294725 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9zfz8"] Nov 04 05:28:47.297955 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:47.297931 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode19ce810_1d70_46db_a8ce_d824a9ffc81e.slice/crio-54e8b758848496b767942024732e9c6aa18697a874cd125465400a2159011c38 WatchSource:0}: Error finding container 54e8b758848496b767942024732e9c6aa18697a874cd125465400a2159011c38: Status 404 returned error can't find the container with id 54e8b758848496b767942024732e9c6aa18697a874cd125465400a2159011c38 Nov 04 05:28:47.312810 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.312787 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9zfz8" event={"ID":"e19ce810-1d70-46db-a8ce-d824a9ffc81e","Type":"ContainerStarted","Data":"54e8b758848496b767942024732e9c6aa18697a874cd125465400a2159011c38"} Nov 04 05:28:47.314279 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.314257 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-skkvk" event={"ID":"c93278df-81cb-480e-9151-ba3056f43d74","Type":"ContainerStarted","Data":"a9ff140e29e1ddc390ee3d6ecd7daa570986685f26af77cc706159c5a7549251"} Nov 04 05:28:47.314377 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.314312 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-skkvk" event={"ID":"c93278df-81cb-480e-9151-ba3056f43d74","Type":"ContainerStarted","Data":"37daf502ec6a620564fe64fcf8e755709761e5b543b611a8c11c693e07680569"} Nov 04 05:28:47.315371 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.315353 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2k9r5" event={"ID":"9f075074-0903-4de4-b247-59a26f5b5f95","Type":"ContainerStarted","Data":"ffc140d9d862cd089d5b3cd81b402ab2707df60ab6400c5a38fef3123c5b7074"} Nov 04 05:28:47.977359 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.977325 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:28:47.977823 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.977801 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:28:47.978171 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.978154 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:28:47.983427 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.983407 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 05:28:47.983531 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.983468 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 05:28:47.983613 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.983577 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 05:28:47.983866 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.983843 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 05:28:47.984003 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.983966 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-kkpcg\"" Nov 04 05:28:47.984179 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.984165 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pqp2n\"" Nov 04 05:28:47.984274 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:47.984256 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zs9xj\"" Nov 04 05:28:48.053078 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.053044 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f5f49d4fc-6cgqw"] Nov 04 05:28:48.055845 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.055823 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.067372 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.066282 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f5f49d4fc-6cgqw"] Nov 04 05:28:48.081612 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.081459 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Nov 04 05:28:48.081612 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.081549 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Nov 04 05:28:48.081612 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.081559 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bgxrs\"" Nov 04 05:28:48.083359 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.083334 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Nov 04 05:28:48.083462 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.083434 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Nov 04 05:28:48.083514 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.083505 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Nov 04 05:28:48.083565 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.083513 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Nov 04 05:28:48.083671 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.083590 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Nov 04 05:28:48.086290 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.086267 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Nov 04 05:28:48.157951 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.157913 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.158123 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.157962 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.158123 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.158016 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7beaf096-fa6b-4233-a68b-1ab974c2e037-tmp\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.158123 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.158041 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.158312 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.158158 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.158312 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.158180 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.158312 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.158202 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnhgf\" (UniqueName: \"kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.158312 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.158272 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.259012 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.258922 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7beaf096-fa6b-4233-a68b-1ab974c2e037-tmp\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.259012 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.258972 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.259214 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.259036 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.259214 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.259063 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.259214 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.259089 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnhgf\" (UniqueName: \"kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.259214 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.259135 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.259214 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.259184 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.259214 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.259208 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.259438 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.259343 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7beaf096-fa6b-4233-a68b-1ab974c2e037-tmp\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.259903 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.259878 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.260988 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.260565 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.260988 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.260565 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.260988 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.260667 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.263464 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.263441 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.263573 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.263468 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.268443 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.268421 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnhgf\" (UniqueName: \"kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.322659 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.322623 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-skkvk" event={"ID":"c93278df-81cb-480e-9151-ba3056f43d74","Type":"ContainerStarted","Data":"a23b30a3fbcf9f9812c5285674caaca869d1fcc66bc3bbaaea63a88cf9e0cc17"} Nov 04 05:28:48.384766 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.384730 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:28:48.667635 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:48.667555 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f5f49d4fc-6cgqw"] Nov 04 05:28:48.672046 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:48.672006 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7beaf096_fa6b_4233_a68b_1ab974c2e037.slice/crio-3f5185ad9371ea13cbae73e1eb02bfa3a76d165dcc331d716aca0b8b2ffdcc12 WatchSource:0}: Error finding container 3f5185ad9371ea13cbae73e1eb02bfa3a76d165dcc331d716aca0b8b2ffdcc12: Status 404 returned error can't find the container with id 3f5185ad9371ea13cbae73e1eb02bfa3a76d165dcc331d716aca0b8b2ffdcc12 Nov 04 05:28:49.328546 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:49.328462 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f5f49d4fc-6cgqw" event={"ID":"7beaf096-fa6b-4233-a68b-1ab974c2e037","Type":"ContainerStarted","Data":"3f5185ad9371ea13cbae73e1eb02bfa3a76d165dcc331d716aca0b8b2ffdcc12"} Nov 04 05:28:50.149534 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.149493 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd"] Nov 04 05:28:50.151912 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.151894 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:28:50.159113 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.159070 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-qxcbb\"" Nov 04 05:28:50.161060 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.161043 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Nov 04 05:28:50.161204 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.161184 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Nov 04 05:28:50.163120 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.162447 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 05:28:50.163120 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.162508 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 05:28:50.163120 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.162688 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 04 05:28:50.166812 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.166791 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dzs66"] Nov 04 05:28:50.169477 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.169187 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd"] Nov 04 05:28:50.169636 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.169618 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.173943 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.172587 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647"] Nov 04 05:28:50.175196 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.175039 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.188975 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.188954 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647"] Nov 04 05:28:50.244688 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.244591 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Nov 04 05:28:50.244688 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.244654 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Nov 04 05:28:50.244954 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.244687 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Nov 04 05:28:50.244954 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.244688 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pwkwp\"" Nov 04 05:28:50.244954 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.244892 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-62jkh\"" Nov 04 05:28:50.244954 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.244889 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Nov 04 05:28:50.262291 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.262249 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Nov 04 05:28:50.262447 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.262299 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Nov 04 05:28:50.273604 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.273573 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.273737 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.273609 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.273737 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.273641 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-metrics-client-ca\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.273858 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.273772 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-root\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.273858 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.273806 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.273858 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.273836 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.273996 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.273858 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/137f70c1-c852-4c5f-823f-b8f7ed565c00-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:28:50.273996 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.273877 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/137f70c1-c852-4c5f-823f-b8f7ed565c00-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:28:50.273996 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.273899 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.273996 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.273918 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c76c8\" (UniqueName: \"kubernetes.io/projected/137f70c1-c852-4c5f-823f-b8f7ed565c00-kube-api-access-c76c8\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:28:50.273996 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.273931 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-sys\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.273996 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.273962 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-wtmp\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.273996 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.273980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddw4d\" (UniqueName: \"kubernetes.io/projected/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-api-access-ddw4d\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.273996 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.273996 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-textfile\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.274289 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.274010 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-accelerators-collector-config\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.274289 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.274041 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-tls\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.274289 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.274061 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/137f70c1-c852-4c5f-823f-b8f7ed565c00-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:28:50.274289 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.274080 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlrrj\" (UniqueName: \"kubernetes.io/projected/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-kube-api-access-wlrrj\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.274289 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.274136 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.333689 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.333652 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9zfz8" event={"ID":"e19ce810-1d70-46db-a8ce-d824a9ffc81e","Type":"ContainerStarted","Data":"dfbec04fce4b55a0a0b31122f1c92abe2486bd7a756e3248c7e247972eb599af"} Nov 04 05:28:50.334125 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.333745 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9zfz8" event={"ID":"e19ce810-1d70-46db-a8ce-d824a9ffc81e","Type":"ContainerStarted","Data":"d638e37d5591a11fe810d9ecf711e0e24233fafca8c88c83db28d7546a596e0a"} Nov 04 05:28:50.334125 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.333798 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9zfz8" Nov 04 05:28:50.336132 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.336107 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-skkvk" event={"ID":"c93278df-81cb-480e-9151-ba3056f43d74","Type":"ContainerStarted","Data":"587849de03694a115a3c8e0f530cbfc994a7e886ede5a6ab7c7d28b2320ed7ab"} Nov 04 05:28:50.350610 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.350565 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-9zfz8" podStartSLOduration=2.383865609 podStartE2EDuration="4.350551429s" totalImagesPullingTime="1.96668582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:50.349666861 +0000 UTC m=+48.037496490" watchObservedRunningTime="2026-11-04 05:28:50.350551429 +0000 UTC m=+48.038381053" Nov 04 05:28:50.367134 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.367084 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-skkvk" podStartSLOduration=2.07361466 podStartE2EDuration="4.367066706s" totalImagesPullingTime="2.293452046s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:28:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:50.366328163 +0000 UTC m=+48.054157791" watchObservedRunningTime="2026-11-04 05:28:50.367066706 +0000 UTC m=+48.054896334" Nov 04 05:28:50.375266 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375241 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-root\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.375374 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375280 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.375374 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375290 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-root\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.375374 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375306 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.375374 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375325 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/137f70c1-c852-4c5f-823f-b8f7ed565c00-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:28:50.375374 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375346 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/137f70c1-c852-4c5f-823f-b8f7ed565c00-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:28:50.377926 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375663 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.377926 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375707 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c76c8\" (UniqueName: \"kubernetes.io/projected/137f70c1-c852-4c5f-823f-b8f7ed565c00-kube-api-access-c76c8\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:28:50.377926 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375737 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-sys\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.377926 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375770 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-wtmp\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.377926 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375804 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddw4d\" (UniqueName: \"kubernetes.io/projected/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-api-access-ddw4d\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.377926 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375869 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.377926 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375881 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-textfile\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.377926 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375908 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-accelerators-collector-config\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.377926 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375961 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-tls\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.377926 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375973 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-wtmp\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.377926 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.375991 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/137f70c1-c852-4c5f-823f-b8f7ed565c00-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:28:50.377926 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.376020 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlrrj\" (UniqueName: \"kubernetes.io/projected/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-kube-api-access-wlrrj\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.377926 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.376064 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.377926 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.376097 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.377926 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.376119 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.377926 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.376149 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-metrics-client-ca\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.378767 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.376269 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-sys\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.378767 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.376819 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-metrics-client-ca\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.378767 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.376900 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/137f70c1-c852-4c5f-823f-b8f7ed565c00-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:28:50.378767 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.377103 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.378767 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.377744 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.379806 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.379440 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/137f70c1-c852-4c5f-823f-b8f7ed565c00-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:28:50.379806 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.379653 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.379806 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.379766 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-textfile\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.380003 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.379860 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-accelerators-collector-config\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.380148 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.380124 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-tls\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.380779 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.380761 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/137f70c1-c852-4c5f-823f-b8f7ed565c00-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:28:50.382280 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.382256 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.382724 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.382699 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.383520 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.383497 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddw4d\" (UniqueName: \"kubernetes.io/projected/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-api-access-ddw4d\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.383948 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.383929 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c76c8\" (UniqueName: \"kubernetes.io/projected/137f70c1-c852-4c5f-823f-b8f7ed565c00-kube-api-access-c76c8\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:28:50.385127 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.385109 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlrrj\" (UniqueName: \"kubernetes.io/projected/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-kube-api-access-wlrrj\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.467825 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.467784 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:28:50.491285 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.491057 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:28:50.500645 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.500621 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:28:50.506477 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:50.505347 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7374bdb0_a62f_4b1c_9d14_e1097c851ff0.slice/crio-579c3d33f20c142fe1026324599c72f7c34a4f83fd3677b35a2ececb10451dba WatchSource:0}: Error finding container 579c3d33f20c142fe1026324599c72f7c34a4f83fd3677b35a2ececb10451dba: Status 404 returned error can't find the container with id 579c3d33f20c142fe1026324599c72f7c34a4f83fd3677b35a2ececb10451dba Nov 04 05:28:50.755698 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.755621 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd"] Nov 04 05:28:50.758689 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:50.758665 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod137f70c1_c852_4c5f_823f_b8f7ed565c00.slice/crio-f1621b0505a7ee1a889b867df2295e11c7af927f0b50f6eaab9619290dcdfd65 WatchSource:0}: Error finding container f1621b0505a7ee1a889b867df2295e11c7af927f0b50f6eaab9619290dcdfd65: Status 404 returned error can't find the container with id f1621b0505a7ee1a889b867df2295e11c7af927f0b50f6eaab9619290dcdfd65 Nov 04 05:28:50.794402 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:50.794377 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647"] Nov 04 05:28:50.798135 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:50.798104 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddeb2d26_5ccb_48c2_9859_09fb6dfe5e15.slice/crio-de37f3c4cee214cc69bffc454ecda72e0f9142c2af060ccde47e2e192b378ac7 WatchSource:0}: Error finding container de37f3c4cee214cc69bffc454ecda72e0f9142c2af060ccde47e2e192b378ac7: Status 404 returned error can't find the container with id de37f3c4cee214cc69bffc454ecda72e0f9142c2af060ccde47e2e192b378ac7 Nov 04 05:28:51.232194 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.232154 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:28:51.235153 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.235124 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.239795 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.239769 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Nov 04 05:28:51.239900 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.239850 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Nov 04 05:28:51.240020 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.239912 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Nov 04 05:28:51.240020 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.239969 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Nov 04 05:28:51.240020 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.240004 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Nov 04 05:28:51.240148 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.240044 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Nov 04 05:28:51.240148 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.240054 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Nov 04 05:28:51.240148 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.240105 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Nov 04 05:28:51.240299 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.240150 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tbvlr\"" Nov 04 05:28:51.240299 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.240167 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Nov 04 05:28:51.250958 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.250882 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:28:51.343059 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.343025 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2k9r5" event={"ID":"9f075074-0903-4de4-b247-59a26f5b5f95","Type":"ContainerStarted","Data":"e8d34532cef96d67d2b20c808fd9cd82f6f0696c0c31edb6cdbf9fee5befa07c"} Nov 04 05:28:51.346192 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.346165 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" event={"ID":"137f70c1-c852-4c5f-823f-b8f7ed565c00","Type":"ContainerStarted","Data":"e47936bd1a34752dfa31d2a4ead67e21f4a204e2865efedfc17e70d504c0a3db"} Nov 04 05:28:51.346387 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.346244 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" event={"ID":"137f70c1-c852-4c5f-823f-b8f7ed565c00","Type":"ContainerStarted","Data":"b0a6c0b57ba19d0b99bc626d6722b30337de1ede18336a2558b07395b94a5c2f"} Nov 04 05:28:51.346387 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.346277 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" event={"ID":"137f70c1-c852-4c5f-823f-b8f7ed565c00","Type":"ContainerStarted","Data":"f1621b0505a7ee1a889b867df2295e11c7af927f0b50f6eaab9619290dcdfd65"} Nov 04 05:28:51.347950 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.347921 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" event={"ID":"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15","Type":"ContainerStarted","Data":"de37f3c4cee214cc69bffc454ecda72e0f9142c2af060ccde47e2e192b378ac7"} Nov 04 05:28:51.349870 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.349806 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dzs66" event={"ID":"7374bdb0-a62f-4b1c-9d14-e1097c851ff0","Type":"ContainerStarted","Data":"579c3d33f20c142fe1026324599c72f7c34a4f83fd3677b35a2ececb10451dba"} Nov 04 05:28:51.383847 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.382925 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.383847 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.382974 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.383847 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.383020 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.383847 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.383048 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.383847 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.383081 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.383847 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.383120 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-config-out\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.383847 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.383158 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.383847 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.383213 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-config-volume\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.383847 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.383275 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-web-config\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.383847 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.383305 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.383847 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.383335 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.383847 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.383358 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28fm9\" (UniqueName: \"kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-kube-api-access-28fm9\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.383847 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.383405 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.484404 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.484326 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.484404 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.484376 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.484612 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.484416 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.484612 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.484434 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.484612 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.484478 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.484612 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.484524 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-config-out\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.484612 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.484552 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.484861 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.484641 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-config-volume\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.484861 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.484658 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-web-config\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.484861 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.484680 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.484861 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.484699 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.484861 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.484713 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28fm9\" (UniqueName: \"kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-kube-api-access-28fm9\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.484861 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.484754 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.489070 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.487882 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.489070 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.488138 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.489070 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.488741 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.491711 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.491685 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.492162 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.492142 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.492724 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.492681 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-config-volume\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.493214 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.493172 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.494333 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.494306 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.496744 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.496703 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28fm9\" (UniqueName: \"kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-kube-api-access-28fm9\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.497540 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.497504 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.498740 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.498719 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-config-out\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.499196 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.499171 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-web-config\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.499383 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.499340 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.556820 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.556786 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.864533 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.864488 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2k9r5" podStartSLOduration=2.740496289 podStartE2EDuration="5.864470373s" totalImagesPullingTime="3.123974084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:51.360626117 +0000 UTC m=+49.048455747" watchObservedRunningTime="2026-11-04 05:28:51.864470373 +0000 UTC m=+49.552300002" Nov 04 05:28:51.865394 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:51.865287 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:28:51.888289 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:51.888255 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcd56353_442a_468a_88b7_6033c8686386.slice/crio-9c8898951746b209c4c5d3ff9c217388d6a6b5bbbed8bb14d7ff0e46c6c609be WatchSource:0}: Error finding container 9c8898951746b209c4c5d3ff9c217388d6a6b5bbbed8bb14d7ff0e46c6c609be: Status 404 returned error can't find the container with id 9c8898951746b209c4c5d3ff9c217388d6a6b5bbbed8bb14d7ff0e46c6c609be Nov 04 05:28:52.230946 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.230911 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-99867478d-zstlh"] Nov 04 05:28:52.233106 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.233090 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.239289 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.238654 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Nov 04 05:28:52.239289 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.238700 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Nov 04 05:28:52.239289 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.238735 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Nov 04 05:28:52.239289 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.238736 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Nov 04 05:28:52.239289 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.238810 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-h4l2m\"" Nov 04 05:28:52.239289 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.238810 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Nov 04 05:28:52.239289 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.238934 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-263s2l7uf16b\"" Nov 04 05:28:52.245535 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.245509 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-99867478d-zstlh"] Nov 04 05:28:52.358727 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.358697 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f5f49d4fc-6cgqw" event={"ID":"7beaf096-fa6b-4233-a68b-1ab974c2e037","Type":"ContainerStarted","Data":"97d36e0f28f1315da3f503de9b66ef3dd55a16de761a7a8fcb120dbe586818fd"} Nov 04 05:28:52.360817 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.360790 2579 generic.go:356] "Generic (PLEG): container finished" podID="7374bdb0-a62f-4b1c-9d14-e1097c851ff0" containerID="161adae46962bdf9e89da45baba3fca1a98ecb0afc296dc3bb98407df913995b" exitCode=0 Nov 04 05:28:52.361113 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.360896 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dzs66" event={"ID":"7374bdb0-a62f-4b1c-9d14-e1097c851ff0","Type":"ContainerDied","Data":"161adae46962bdf9e89da45baba3fca1a98ecb0afc296dc3bb98407df913995b"} Nov 04 05:28:52.362516 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.362488 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dcd56353-442a-468a-88b7-6033c8686386","Type":"ContainerStarted","Data":"9c8898951746b209c4c5d3ff9c217388d6a6b5bbbed8bb14d7ff0e46c6c609be"} Nov 04 05:28:52.366188 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.365961 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" event={"ID":"137f70c1-c852-4c5f-823f-b8f7ed565c00","Type":"ContainerStarted","Data":"db54e1c4af212ae1b2125c14d83fef6947468dbf10bb954604d7f829bcdea80e"} Nov 04 05:28:52.382005 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.381442 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5f5f49d4fc-6cgqw" podStartSLOduration=0.982501375 podStartE2EDuration="4.381425507s" totalImagesPullingTime="3.398924132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:28:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:52.379298556 +0000 UTC m=+50.067128205" watchObservedRunningTime="2026-11-04 05:28:52.381425507 +0000 UTC m=+50.069255136" Nov 04 05:28:52.388329 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.388177 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" event={"ID":"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15","Type":"ContainerStarted","Data":"dbbc926d6fdf5a7e9ef7d6ce662d3890ba207f6a01db6752a4f37eb183be5cc2"} Nov 04 05:28:52.388329 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.388281 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" event={"ID":"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15","Type":"ContainerStarted","Data":"26f7ba746c748155e822a7c454279bb17943b8208f64dc03c7153e3a883011dc"} Nov 04 05:28:52.394182 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.394152 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.394414 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.394394 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.394534 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.394519 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-tls\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.394682 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.394669 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dmt9\" (UniqueName: \"kubernetes.io/projected/e382c8f0-2b69-4271-8098-8328ac9b53ed-kube-api-access-9dmt9\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.395102 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.394765 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.395102 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.394793 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e382c8f0-2b69-4271-8098-8328ac9b53ed-tmp\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.395102 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.394816 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-grpc-tls\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.395102 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.394874 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.395102 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.394910 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e382c8f0-2b69-4271-8098-8328ac9b53ed-metrics-client-ca\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.412417 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.412373 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" podStartSLOduration=1.22353309 podStartE2EDuration="2.41235933s" totalImagesPullingTime="1.18882624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:28:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:52.411143571 +0000 UTC m=+50.098973200" watchObservedRunningTime="2026-11-04 05:28:52.41235933 +0000 UTC m=+50.100188957" Nov 04 05:28:52.495612 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.495592 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.496524 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.496507 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e382c8f0-2b69-4271-8098-8328ac9b53ed-metrics-client-ca\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.496743 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.496731 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.496829 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.496819 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.496911 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.496903 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-tls\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.496964 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.496944 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e382c8f0-2b69-4271-8098-8328ac9b53ed-metrics-client-ca\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.497420 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.497093 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dmt9\" (UniqueName: \"kubernetes.io/projected/e382c8f0-2b69-4271-8098-8328ac9b53ed-kube-api-access-9dmt9\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.497420 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.497130 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.497420 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.497157 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e382c8f0-2b69-4271-8098-8328ac9b53ed-tmp\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.497420 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.497183 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-grpc-tls\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.499604 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.499104 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.499604 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.499562 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-tls\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.500406 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.500313 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e382c8f0-2b69-4271-8098-8328ac9b53ed-tmp\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.500493 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.500444 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.501834 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.501800 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.501999 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.501852 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.502469 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.502453 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-grpc-tls\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.508820 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.508798 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dmt9\" (UniqueName: \"kubernetes.io/projected/e382c8f0-2b69-4271-8098-8328ac9b53ed-kube-api-access-9dmt9\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.550080 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.549854 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:52.826698 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:52.826661 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-99867478d-zstlh"] Nov 04 05:28:52.829488 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:52.829457 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode382c8f0_2b69_4271_8098_8328ac9b53ed.slice/crio-4a065ef3d59ba12dd48cf4231807c8c00afb5d135f136d7dabfe7b53b2356aea WatchSource:0}: Error finding container 4a065ef3d59ba12dd48cf4231807c8c00afb5d135f136d7dabfe7b53b2356aea: Status 404 returned error can't find the container with id 4a065ef3d59ba12dd48cf4231807c8c00afb5d135f136d7dabfe7b53b2356aea Nov 04 05:28:53.395083 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:53.394965 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dzs66" event={"ID":"7374bdb0-a62f-4b1c-9d14-e1097c851ff0","Type":"ContainerStarted","Data":"e856dde27c06ea73cefce4bff7de70f2647273eb1cd230904bf487fef5f1f18e"} Nov 04 05:28:53.395083 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:53.395031 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dzs66" event={"ID":"7374bdb0-a62f-4b1c-9d14-e1097c851ff0","Type":"ContainerStarted","Data":"0af882568be1057633ed4cc8af37db5a14e57a8e0cd4c452780abce4c5b458f2"} Nov 04 05:28:53.397135 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:53.397108 2579 generic.go:356] "Generic (PLEG): container finished" podID="dcd56353-442a-468a-88b7-6033c8686386" containerID="b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86" exitCode=0 Nov 04 05:28:53.397351 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:53.397142 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dcd56353-442a-468a-88b7-6033c8686386","Type":"ContainerDied","Data":"b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86"} Nov 04 05:28:53.398633 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:53.398608 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-99867478d-zstlh" event={"ID":"e382c8f0-2b69-4271-8098-8328ac9b53ed","Type":"ContainerStarted","Data":"4a065ef3d59ba12dd48cf4231807c8c00afb5d135f136d7dabfe7b53b2356aea"} Nov 04 05:28:53.401035 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:53.401012 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" event={"ID":"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15","Type":"ContainerStarted","Data":"599bae6730e305f238e906fe6404a140d7486046f64d2c34f490f209354ec862"} Nov 04 05:28:53.414176 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:53.414127 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dzs66" podStartSLOduration=2.139061274 podStartE2EDuration="3.414111018s" totalImagesPullingTime="1.1949786s" totalInitContainerRuntime="80.071144ms" isStatefulPod=true podCreationTimestamp="2026-11-04 05:28:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:53.413139844 +0000 UTC m=+51.100969471" watchObservedRunningTime="2026-11-04 05:28:53.414111018 +0000 UTC m=+51.101940647" Nov 04 05:28:53.457959 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:53.457917 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" podStartSLOduration=2.003602381 podStartE2EDuration="3.457841352s" totalImagesPullingTime="1.454238971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:28:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:53.45589515 +0000 UTC m=+51.143724780" watchObservedRunningTime="2026-11-04 05:28:53.457841352 +0000 UTC m=+51.145670980" Nov 04 05:28:54.556261 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.556212 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-765454fc56-b5f6j"] Nov 04 05:28:54.558492 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.558469 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.568549 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.568438 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Nov 04 05:28:54.568549 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.568521 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Nov 04 05:28:54.569791 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.569739 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-765454fc56-b5f6j"] Nov 04 05:28:54.605381 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.605353 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-lllmb\"" Nov 04 05:28:54.605381 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.605354 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Nov 04 05:28:54.605579 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.605353 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Nov 04 05:28:54.605579 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.605402 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-35ajntqbki0o8\"" Nov 04 05:28:54.725793 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.725753 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-secret-metrics-server-client-certs\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.725951 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.725801 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32bea234-f8f1-41fa-ac2f-bea97c729d10-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.725951 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.725937 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/32bea234-f8f1-41fa-ac2f-bea97c729d10-audit-log\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.726034 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.725967 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-client-ca-bundle\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.726034 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.725993 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-secret-metrics-server-tls\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.726115 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.726094 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/32bea234-f8f1-41fa-ac2f-bea97c729d10-metrics-server-audit-profiles\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.726145 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.726115 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrnng\" (UniqueName: \"kubernetes.io/projected/32bea234-f8f1-41fa-ac2f-bea97c729d10-kube-api-access-mrnng\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.827642 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.827024 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/32bea234-f8f1-41fa-ac2f-bea97c729d10-metrics-server-audit-profiles\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.827642 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.827075 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrnng\" (UniqueName: \"kubernetes.io/projected/32bea234-f8f1-41fa-ac2f-bea97c729d10-kube-api-access-mrnng\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.827642 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.827120 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-secret-metrics-server-client-certs\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.827642 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.827147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32bea234-f8f1-41fa-ac2f-bea97c729d10-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.827642 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.827187 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/32bea234-f8f1-41fa-ac2f-bea97c729d10-audit-log\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.827642 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.827210 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-client-ca-bundle\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.827642 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.827251 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-secret-metrics-server-tls\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.828431 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.828061 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/32bea234-f8f1-41fa-ac2f-bea97c729d10-metrics-server-audit-profiles\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.828884 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.828862 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32bea234-f8f1-41fa-ac2f-bea97c729d10-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.829023 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.828998 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/32bea234-f8f1-41fa-ac2f-bea97c729d10-audit-log\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.830916 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.830872 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-secret-metrics-server-client-certs\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.832480 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.832460 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-client-ca-bundle\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.835065 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.835013 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-secret-metrics-server-tls\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.836012 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.835979 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrnng\" (UniqueName: \"kubernetes.io/projected/32bea234-f8f1-41fa-ac2f-bea97c729d10-kube-api-access-mrnng\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.873170 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.873145 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:28:54.953632 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.953591 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-4s58k"] Nov 04 05:28:54.960905 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.960535 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" Nov 04 05:28:54.965520 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.965381 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-4s58k"] Nov 04 05:28:54.965667 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.965645 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Nov 04 05:28:54.965805 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:54.965789 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2jgc2\"" Nov 04 05:28:55.028986 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.028950 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52797b3b-80b4-443e-aa1d-158430ee739e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-4s58k\" (UID: \"52797b3b-80b4-443e-aa1d-158430ee739e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" Nov 04 05:28:55.029135 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.029084 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjz6h\" (UniqueName: \"kubernetes.io/projected/52797b3b-80b4-443e-aa1d-158430ee739e-kube-api-access-sjz6h\") pod \"monitoring-plugin-77f4f76594-4s58k\" (UID: \"52797b3b-80b4-443e-aa1d-158430ee739e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" Nov 04 05:28:55.130306 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.130259 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52797b3b-80b4-443e-aa1d-158430ee739e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-4s58k\" (UID: \"52797b3b-80b4-443e-aa1d-158430ee739e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" Nov 04 05:28:55.130478 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.130349 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjz6h\" (UniqueName: \"kubernetes.io/projected/52797b3b-80b4-443e-aa1d-158430ee739e-kube-api-access-sjz6h\") pod \"monitoring-plugin-77f4f76594-4s58k\" (UID: \"52797b3b-80b4-443e-aa1d-158430ee739e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" Nov 04 05:28:55.132746 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.132723 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52797b3b-80b4-443e-aa1d-158430ee739e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-4s58k\" (UID: \"52797b3b-80b4-443e-aa1d-158430ee739e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" Nov 04 05:28:55.138121 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.138094 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjz6h\" (UniqueName: \"kubernetes.io/projected/52797b3b-80b4-443e-aa1d-158430ee739e-kube-api-access-sjz6h\") pod \"monitoring-plugin-77f4f76594-4s58k\" (UID: \"52797b3b-80b4-443e-aa1d-158430ee739e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" Nov 04 05:28:55.155498 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.155474 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-765454fc56-b5f6j"] Nov 04 05:28:55.158495 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:55.158467 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32bea234_f8f1_41fa_ac2f_bea97c729d10.slice/crio-ca79ba46ecc6a6f60d1cc0136524237031a249b5f03a020926b52a6d7576b16d WatchSource:0}: Error finding container ca79ba46ecc6a6f60d1cc0136524237031a249b5f03a020926b52a6d7576b16d: Status 404 returned error can't find the container with id ca79ba46ecc6a6f60d1cc0136524237031a249b5f03a020926b52a6d7576b16d Nov 04 05:28:55.273648 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.273621 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" Nov 04 05:28:55.412610 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.412188 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-99867478d-zstlh" event={"ID":"e382c8f0-2b69-4271-8098-8328ac9b53ed","Type":"ContainerStarted","Data":"22f2740b144506f3fa46798be6b494e49d15ca2c0bc3acc29344c414b8708d84"} Nov 04 05:28:55.412610 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.412298 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-99867478d-zstlh" event={"ID":"e382c8f0-2b69-4271-8098-8328ac9b53ed","Type":"ContainerStarted","Data":"c104117b0baa8018599d5ae098bf21151ed3d2a55acc010d4391b11777759811"} Nov 04 05:28:55.412610 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.412338 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-99867478d-zstlh" event={"ID":"e382c8f0-2b69-4271-8098-8328ac9b53ed","Type":"ContainerStarted","Data":"cc18a0a50b8328e9b749dd2d31038526cd56375ec59588e9f441758917086330"} Nov 04 05:28:55.421686 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.420897 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" event={"ID":"32bea234-f8f1-41fa-ac2f-bea97c729d10","Type":"ContainerStarted","Data":"ca79ba46ecc6a6f60d1cc0136524237031a249b5f03a020926b52a6d7576b16d"} Nov 04 05:28:55.432638 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.432481 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dcd56353-442a-468a-88b7-6033c8686386","Type":"ContainerStarted","Data":"9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78"} Nov 04 05:28:55.432716 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.432657 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dcd56353-442a-468a-88b7-6033c8686386","Type":"ContainerStarted","Data":"114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c"} Nov 04 05:28:55.432750 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.432719 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dcd56353-442a-468a-88b7-6033c8686386","Type":"ContainerStarted","Data":"21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78"} Nov 04 05:28:55.432779 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.432771 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dcd56353-442a-468a-88b7-6033c8686386","Type":"ContainerStarted","Data":"f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7"} Nov 04 05:28:55.549003 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.548974 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-4s58k"] Nov 04 05:28:55.558252 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:55.558200 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52797b3b_80b4_443e_aa1d_158430ee739e.slice/crio-68869ffb949af724efeba8265deadbac63f5df90c9be986b392d98c3e12d09c1 WatchSource:0}: Error finding container 68869ffb949af724efeba8265deadbac63f5df90c9be986b392d98c3e12d09c1: Status 404 returned error can't find the container with id 68869ffb949af724efeba8265deadbac63f5df90c9be986b392d98c3e12d09c1 Nov 04 05:28:55.660583 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.660506 2579 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f5f49d4fc-6cgqw"] Nov 04 05:28:55.688798 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.688761 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6db77f4495-958wf"] Nov 04 05:28:55.691785 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.691765 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.701985 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.701960 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6db77f4495-958wf"] Nov 04 05:28:55.735675 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.735628 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.735810 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.735706 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.735810 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.735787 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.735810 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.735805 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mwlv\" (UniqueName: \"kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.735973 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.735835 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.735973 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.735880 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.735973 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.735914 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.735973 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.735939 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e39328c2-e756-4c99-9d13-6fbdccae91cb-tmp\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.836679 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.836650 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.836791 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.836693 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mwlv\" (UniqueName: \"kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.836791 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.836739 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.836791 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.836785 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.836950 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.836831 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.836950 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.836855 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e39328c2-e756-4c99-9d13-6fbdccae91cb-tmp\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.836950 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.836883 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.836950 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.836924 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.837657 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.837634 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.837765 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.837653 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e39328c2-e756-4c99-9d13-6fbdccae91cb-tmp\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.837765 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.837640 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.837765 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.837680 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.838173 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.838126 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.839642 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.839619 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.840142 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.840119 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:55.846057 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:55.845558 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mwlv\" (UniqueName: \"kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:56.003977 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.003951 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:28:56.282818 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.282744 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:28:56.285249 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.285203 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6db77f4495-958wf"] Nov 04 05:28:56.289868 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:56.289842 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode39328c2_e756_4c99_9d13_6fbdccae91cb.slice/crio-88dbe066e70488397c0c8a44b49766afe9dd1661739d3a6f1b29d418b982ea80 WatchSource:0}: Error finding container 88dbe066e70488397c0c8a44b49766afe9dd1661739d3a6f1b29d418b982ea80: Status 404 returned error can't find the container with id 88dbe066e70488397c0c8a44b49766afe9dd1661739d3a6f1b29d418b982ea80 Nov 04 05:28:56.441039 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.441008 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:28:56.444106 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.444082 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.444545 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.444517 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db77f4495-958wf" event={"ID":"e39328c2-e756-4c99-9d13-6fbdccae91cb","Type":"ContainerStarted","Data":"38c705efee78d36182ed89aec77d6ceb65252e4482cc100e522c6bfa374ed24e"} Nov 04 05:28:56.444677 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.444560 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db77f4495-958wf" event={"ID":"e39328c2-e756-4c99-9d13-6fbdccae91cb","Type":"ContainerStarted","Data":"88dbe066e70488397c0c8a44b49766afe9dd1661739d3a6f1b29d418b982ea80"} Nov 04 05:28:56.446659 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.446608 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" event={"ID":"52797b3b-80b4-443e-aa1d-158430ee739e","Type":"ContainerStarted","Data":"68869ffb949af724efeba8265deadbac63f5df90c9be986b392d98c3e12d09c1"} Nov 04 05:28:56.447975 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.447879 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Nov 04 05:28:56.447975 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.447879 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Nov 04 05:28:56.448437 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.448420 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-teufh2l7p2i9\"" Nov 04 05:28:56.448540 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.448471 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Nov 04 05:28:56.448540 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.448528 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Nov 04 05:28:56.448650 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.448601 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8w4gl\"" Nov 04 05:28:56.448650 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.448625 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Nov 04 05:28:56.448725 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.448650 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Nov 04 05:28:56.448725 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.448690 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Nov 04 05:28:56.448792 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.448731 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Nov 04 05:28:56.448792 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.448764 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Nov 04 05:28:56.448857 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.448810 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Nov 04 05:28:56.448857 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.448823 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Nov 04 05:28:56.448857 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.448852 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Nov 04 05:28:56.451633 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.451609 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dcd56353-442a-468a-88b7-6033c8686386","Type":"ContainerStarted","Data":"bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25"} Nov 04 05:28:56.451699 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.451675 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dcd56353-442a-468a-88b7-6033c8686386","Type":"ContainerStarted","Data":"601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b"} Nov 04 05:28:56.452397 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.452377 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Nov 04 05:28:56.452586 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.452565 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Nov 04 05:28:56.459949 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.459926 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:28:56.469551 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.469532 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-99867478d-zstlh" event={"ID":"e382c8f0-2b69-4271-8098-8328ac9b53ed","Type":"ContainerStarted","Data":"0fdc2f7ccab6b4b8eb32c0986cf51897861275db2c62bd96e9206f72da3dfc9e"} Nov 04 05:28:56.470258 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.470236 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-99867478d-zstlh" event={"ID":"e382c8f0-2b69-4271-8098-8328ac9b53ed","Type":"ContainerStarted","Data":"01d170667bc0a5d6f3f3eca6ce3c50a5ecdfb333fc2118b61f46993cf58b3879"} Nov 04 05:28:56.470401 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.470385 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-99867478d-zstlh" event={"ID":"e382c8f0-2b69-4271-8098-8328ac9b53ed","Type":"ContainerStarted","Data":"ded3b3d670e2a7447f468d782095440949cf785a48d7a40d2e22c5f179b8ee9f"} Nov 04 05:28:56.470486 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.470477 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:28:56.501875 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.501819 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-99867478d-zstlh" podStartSLOduration=1.786516476 podStartE2EDuration="4.501800513s" totalImagesPullingTime="2.715284037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:28:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:56.500201202 +0000 UTC m=+54.188030830" watchObservedRunningTime="2026-11-04 05:28:56.501800513 +0000 UTC m=+54.189630176" Nov 04 05:28:56.517236 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.517191 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6db77f4495-958wf" podStartSLOduration=1.5171793409999998 podStartE2EDuration="1.517179341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:28:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:56.516046319 +0000 UTC m=+54.203875949" watchObservedRunningTime="2026-11-04 05:28:56.517179341 +0000 UTC m=+54.205008963" Nov 04 05:28:56.541327 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.541243 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.189371636 podStartE2EDuration="5.541216622s" totalImagesPullingTime="3.335295104s" totalInitContainerRuntime="16.549882ms" isStatefulPod=false podCreationTimestamp="2026-11-04 05:28:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:56.539767396 +0000 UTC m=+54.227597025" watchObservedRunningTime="2026-11-04 05:28:56.541216622 +0000 UTC m=+54.229046300" Nov 04 05:28:56.546854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.544721 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-config-out\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.546854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.544787 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.546854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.544825 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-config\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.546854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.544856 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.546854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.544880 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.546854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.544899 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d92sv\" (UniqueName: \"kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-kube-api-access-d92sv\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.546854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.545009 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.546854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.545034 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.546854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.545129 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.546854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.545345 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.546854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.545541 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.546854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.546112 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.546854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.546171 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.546854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.546665 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-web-config\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.546854 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.546699 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.548035 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.547251 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.548035 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.547308 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.548035 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.547349 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.548035 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.547369 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.548035 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.547458 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.648976 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.648935 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.649471 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.648982 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.649471 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649035 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.649471 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649078 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.649471 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649107 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.649471 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649164 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.649471 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649188 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.649471 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649217 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-web-config\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.649471 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649264 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.649471 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649308 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.649471 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649335 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.649471 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649360 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.649471 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649382 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.649471 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649407 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.649471 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649453 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-config-out\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.649471 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649474 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.650133 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649508 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-config\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.650133 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649541 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.650133 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649565 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.650133 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649588 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d92sv\" (UniqueName: \"kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-kube-api-access-d92sv\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.650133 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.649865 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.650405 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.650189 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.650405 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.650189 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.651363 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.651327 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.651363 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.651348 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.652617 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.652592 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.653156 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.653130 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.653795 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.653768 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.653877 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.653810 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.654042 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.654019 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-config-out\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.654428 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.654402 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.654518 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.654468 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.654814 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.654792 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.656136 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.655695 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-web-config\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.656136 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.655831 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.656136 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.655929 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.656136 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.656095 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.657139 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.657108 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.657739 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.657714 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d92sv\" (UniqueName: \"kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-kube-api-access-d92sv\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.657860 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.657829 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-config\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.770678 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:56.770641 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.060343 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:57.060310 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:28:57.064381 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:28:57.064331 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode402f928_dd6c_4f77_acbb_94b57e6b5188.slice/crio-75fd2d220aed24cc4be18d597bc21fe0ae142ab309de07a195021252a7217aa6 WatchSource:0}: Error finding container 75fd2d220aed24cc4be18d597bc21fe0ae142ab309de07a195021252a7217aa6: Status 404 returned error can't find the container with id 75fd2d220aed24cc4be18d597bc21fe0ae142ab309de07a195021252a7217aa6 Nov 04 05:28:57.475100 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:57.474997 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" event={"ID":"52797b3b-80b4-443e-aa1d-158430ee739e","Type":"ContainerStarted","Data":"1d426702a38a67d93e3be7420f54dfcdf0e4cbed57dfab3820367ce5b556bde5"} Nov 04 05:28:57.475268 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:57.475257 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" Nov 04 05:28:57.476738 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:57.476712 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" event={"ID":"32bea234-f8f1-41fa-ac2f-bea97c729d10","Type":"ContainerStarted","Data":"161b05e8e528afbb749e9a1990fd418767c7e7df35119a3a64a16677792e1f80"} Nov 04 05:28:57.478403 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:57.478378 2579 generic.go:356] "Generic (PLEG): container finished" podID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerID="3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb" exitCode=0 Nov 04 05:28:57.478546 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:57.478477 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e402f928-dd6c-4f77-acbb-94b57e6b5188","Type":"ContainerDied","Data":"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb"} Nov 04 05:28:57.478546 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:57.478529 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e402f928-dd6c-4f77-acbb-94b57e6b5188","Type":"ContainerStarted","Data":"75fd2d220aed24cc4be18d597bc21fe0ae142ab309de07a195021252a7217aa6"} Nov 04 05:28:57.481003 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:57.480986 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" Nov 04 05:28:57.490069 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:57.489708 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" podStartSLOduration=2.000135851 podStartE2EDuration="3.489693015s" totalImagesPullingTime="1.489557164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:28:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:57.488967939 +0000 UTC m=+55.176797568" watchObservedRunningTime="2026-11-04 05:28:57.489693015 +0000 UTC m=+55.177522648" Nov 04 05:28:57.531716 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:57.531672 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" podStartSLOduration=1.876486507 podStartE2EDuration="3.531660988s" totalImagesPullingTime="1.655174481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:28:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:57.530281073 +0000 UTC m=+55.218110904" watchObservedRunningTime="2026-11-04 05:28:57.531660988 +0000 UTC m=+55.219490616" Nov 04 05:28:58.385645 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:28:58.385611 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:00.352590 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:00.352561 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9zfz8" Nov 04 05:29:00.883376 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:00.883348 2579 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6db77f4495-958wf"] Nov 04 05:29:01.498360 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:01.498322 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e402f928-dd6c-4f77-acbb-94b57e6b5188","Type":"ContainerStarted","Data":"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6"} Nov 04 05:29:01.498741 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:01.498460 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e402f928-dd6c-4f77-acbb-94b57e6b5188","Type":"ContainerStarted","Data":"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df"} Nov 04 05:29:01.498741 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:01.498508 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e402f928-dd6c-4f77-acbb-94b57e6b5188","Type":"ContainerStarted","Data":"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977"} Nov 04 05:29:01.498741 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:01.498544 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e402f928-dd6c-4f77-acbb-94b57e6b5188","Type":"ContainerStarted","Data":"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff"} Nov 04 05:29:01.498741 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:01.498579 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e402f928-dd6c-4f77-acbb-94b57e6b5188","Type":"ContainerStarted","Data":"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45"} Nov 04 05:29:01.498741 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:01.498615 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e402f928-dd6c-4f77-acbb-94b57e6b5188","Type":"ContainerStarted","Data":"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b"} Nov 04 05:29:01.529463 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:01.529398 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.255767082 podStartE2EDuration="5.529381666s" totalImagesPullingTime="3.253572573s" totalInitContainerRuntime="20.042011ms" isStatefulPod=false podCreationTimestamp="2026-11-04 05:28:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:29:01.528140843 +0000 UTC m=+59.215970471" watchObservedRunningTime="2026-11-04 05:29:01.529381666 +0000 UTC m=+59.217211294" Nov 04 05:29:01.771404 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:01.771330 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:03.488897 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:03.488867 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:06.005153 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:06.005123 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:07.545941 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:07.545897 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:29:07.546322 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:07.545970 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4b8q\" (UniqueName: \"kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q\") pod \"network-check-target-mq5nl\" (UID: \"47be26f2-143e-4b49-824a-415e60f6140b\") " pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:29:07.548832 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:07.548812 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 05:29:07.548911 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:07.548897 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 05:29:07.559153 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:07.559104 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:29:07.559382 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:07.559366 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 05:29:07.569937 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:07.569909 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4b8q\" (UniqueName: \"kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q\") pod \"network-check-target-mq5nl\" (UID: \"47be26f2-143e-4b49-824a-415e60f6140b\") " pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:29:07.646983 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:07.646941 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:29:07.649851 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:07.649833 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 05:29:07.659971 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:07.659951 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:29:07.796251 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:07.796163 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-kkpcg\"" Nov 04 05:29:07.804294 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:07.804276 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:29:07.805760 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:07.805741 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pqp2n\"" Nov 04 05:29:07.810793 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:07.810776 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zs9xj\"" Nov 04 05:29:07.813950 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:07.813931 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:29:07.819199 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:07.819184 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:29:08.086377 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:08.086347 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-69rl9"] Nov 04 05:29:08.089590 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:29:08.089561 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod379bdc95_36d7_47f2_9e17_a96e51da6495.slice/crio-d96ea9b49f192b58243c43e8fafd06a4fd251b6290ffc75d0a5439abbe487bff WatchSource:0}: Error finding container d96ea9b49f192b58243c43e8fafd06a4fd251b6290ffc75d0a5439abbe487bff: Status 404 returned error can't find the container with id d96ea9b49f192b58243c43e8fafd06a4fd251b6290ffc75d0a5439abbe487bff Nov 04 05:29:08.099719 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:08.099700 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-brh9c"] Nov 04 05:29:08.101511 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:29:08.101487 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefbcf64e_3ade_4343_a9c8_bf945b141a02.slice/crio-af6995ea37d42bd2c4eb3ba220696c0b46e67491722cdb5ebbdbde5bad41cfb6 WatchSource:0}: Error finding container af6995ea37d42bd2c4eb3ba220696c0b46e67491722cdb5ebbdbde5bad41cfb6: Status 404 returned error can't find the container with id af6995ea37d42bd2c4eb3ba220696c0b46e67491722cdb5ebbdbde5bad41cfb6 Nov 04 05:29:08.114680 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:08.114656 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mq5nl"] Nov 04 05:29:08.117033 ip-10-0-128-126 kubenswrapper[2579]: W1104 05:29:08.117009 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47be26f2_143e_4b49_824a_415e60f6140b.slice/crio-69524c5d81dbedb21b8902882d183dd54090d0c8bba09f1bb720201a28cc478e WatchSource:0}: Error finding container 69524c5d81dbedb21b8902882d183dd54090d0c8bba09f1bb720201a28cc478e: Status 404 returned error can't find the container with id 69524c5d81dbedb21b8902882d183dd54090d0c8bba09f1bb720201a28cc478e Nov 04 05:29:08.523323 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:08.523238 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-brh9c" event={"ID":"efbcf64e-3ade-4343-a9c8-bf945b141a02","Type":"ContainerStarted","Data":"af6995ea37d42bd2c4eb3ba220696c0b46e67491722cdb5ebbdbde5bad41cfb6"} Nov 04 05:29:08.524289 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:08.524265 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-69rl9" event={"ID":"379bdc95-36d7-47f2-9e17-a96e51da6495","Type":"ContainerStarted","Data":"d96ea9b49f192b58243c43e8fafd06a4fd251b6290ffc75d0a5439abbe487bff"} Nov 04 05:29:08.525300 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:08.525281 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mq5nl" event={"ID":"47be26f2-143e-4b49-824a-415e60f6140b","Type":"ContainerStarted","Data":"69524c5d81dbedb21b8902882d183dd54090d0c8bba09f1bb720201a28cc478e"} Nov 04 05:29:09.543253 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:09.543203 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-brh9c" event={"ID":"efbcf64e-3ade-4343-a9c8-bf945b141a02","Type":"ContainerStarted","Data":"e7e628221319ebd4e04ede93806ec33e546e491ece0184fd5d6e4fac46a47288"} Nov 04 05:29:10.549884 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:10.549842 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-brh9c" event={"ID":"efbcf64e-3ade-4343-a9c8-bf945b141a02","Type":"ContainerStarted","Data":"384b42d4499b26b1271ee1b26c2a9f1ab7fc4a613eca413750de5d8d2baf884d"} Nov 04 05:29:10.570058 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:10.569438 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-brh9c" podStartSLOduration=66.328084239 podStartE2EDuration="1m7.569420762s" totalImagesPullingTime="1.241336523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:29:10.56848402 +0000 UTC m=+68.256313649" watchObservedRunningTime="2026-11-04 05:29:10.569420762 +0000 UTC m=+68.257250393" Nov 04 05:29:11.556098 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:11.556058 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mq5nl" event={"ID":"47be26f2-143e-4b49-824a-415e60f6140b","Type":"ContainerStarted","Data":"61374238757a8c436d4650780689bd5a58251dafd795cd1ff94203298c656802"} Nov 04 05:29:11.557397 ip-10-0-128-126 kubenswrapper[2579]: I1104 05:29:11.556426 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:29:12.979663 ip-10-0-128-126 systemd[1]: Stopping Kubernetes Kubelet... Nov 04 05:29:12.991567 ip-10-0-128-126 systemd[1]: kubelet.service: Deactivated successfully. Nov 04 05:29:12.991732 ip-10-0-128-126 systemd[1]: Stopped Kubernetes Kubelet. Nov 04 05:29:12.991896 ip-10-0-128-126 systemd[1]: kubelet.service: Consumed 4.496s CPU time. Nov 04 05:29:13.002054 ip-10-0-128-126 systemd[1]: Starting Kubernetes Kubelet... Nov 04 05:29:13.074358 ip-10-0-128-126 kubenswrapper[5904]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:29:13.074358 ip-10-0-128-126 kubenswrapper[5904]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 05:29:13.074358 ip-10-0-128-126 kubenswrapper[5904]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:29:13.074358 ip-10-0-128-126 kubenswrapper[5904]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:29:13.076434 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076354 5904 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:29:13.076434 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076432 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:29:13.076434 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076436 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076439 5904 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076442 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076445 5904 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076448 5904 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076451 5904 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076460 5904 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076466 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076470 5904 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076473 5904 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076476 5904 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076480 5904 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076484 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076487 5904 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076490 5904 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076493 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076496 5904 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076499 5904 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076501 5904 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:29:13.076531 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076504 5904 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076507 5904 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076510 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076513 5904 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076516 5904 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076518 5904 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076522 5904 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076524 5904 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076527 5904 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076530 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076533 5904 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076536 5904 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076539 5904 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076542 5904 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076545 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076548 5904 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076551 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076556 5904 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076561 5904 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076566 5904 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:29:13.076987 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076571 5904 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076575 5904 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076579 5904 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076583 5904 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076587 5904 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076590 5904 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076593 5904 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076596 5904 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076599 5904 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076602 5904 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076604 5904 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076607 5904 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076610 5904 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076612 5904 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076616 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076619 5904 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076622 5904 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076626 5904 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076628 5904 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076633 5904 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:29:13.077481 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076636 5904 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076639 5904 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076641 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076644 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076646 5904 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076649 5904 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076652 5904 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076656 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076658 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076661 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076664 5904 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076667 5904 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076670 5904 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076673 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076675 5904 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076678 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076681 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076683 5904 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076686 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076689 5904 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:29:13.077943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076692 5904 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076695 5904 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076697 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076700 5904 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076704 5904 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076706 5904 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076709 5904 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076712 5904 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076714 5904 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076717 5904 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076720 5904 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076723 5904 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076726 5904 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076729 5904 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076732 5904 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076734 5904 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076737 5904 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076739 5904 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076742 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:29:13.078427 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076745 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076749 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076753 5904 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076756 5904 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076759 5904 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.076761 5904 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077515 5904 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077523 5904 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077526 5904 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077529 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077532 5904 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077535 5904 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077538 5904 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077541 5904 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077544 5904 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077548 5904 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077550 5904 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077553 5904 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077556 5904 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077559 5904 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:29:13.078911 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077561 5904 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077564 5904 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077567 5904 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077569 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077572 5904 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077575 5904 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077578 5904 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077581 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077584 5904 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077586 5904 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077589 5904 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077592 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077594 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077597 5904 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077600 5904 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077603 5904 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077605 5904 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077608 5904 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077611 5904 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077613 5904 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:29:13.079393 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077616 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077619 5904 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077622 5904 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077624 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077627 5904 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077630 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077634 5904 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077638 5904 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077641 5904 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077645 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077648 5904 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077651 5904 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077654 5904 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077657 5904 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077659 5904 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077662 5904 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077665 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077667 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077670 5904 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:29:13.079858 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077673 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077675 5904 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077679 5904 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077681 5904 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077684 5904 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077687 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077689 5904 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077692 5904 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077695 5904 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077703 5904 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077706 5904 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077709 5904 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077712 5904 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077715 5904 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077717 5904 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077720 5904 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077723 5904 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077726 5904 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077729 5904 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:29:13.080316 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077731 5904 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077736 5904 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077740 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077743 5904 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077746 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077749 5904 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077752 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077755 5904 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077757 5904 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077760 5904 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077763 5904 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077766 5904 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077768 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077771 5904 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077774 5904 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077776 5904 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077779 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077782 5904 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077784 5904 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:29:13.080761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077787 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077789 5904 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077792 5904 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077795 5904 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077798 5904 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077802 5904 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077805 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077808 5904 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077811 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077814 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077816 5904 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077819 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077822 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077824 5904 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.077827 5904 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078331 5904 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078339 5904 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078342 5904 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078345 5904 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078348 5904 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:29:13.081208 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078351 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078353 5904 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078357 5904 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078359 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078362 5904 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078365 5904 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078368 5904 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078371 5904 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078373 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078376 5904 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078379 5904 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078381 5904 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078384 5904 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078387 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078390 5904 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078393 5904 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078396 5904 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078401 5904 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078406 5904 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078409 5904 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:29:13.081692 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078412 5904 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078416 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078418 5904 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078422 5904 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078425 5904 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078428 5904 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078431 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078434 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078436 5904 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078439 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078442 5904 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078446 5904 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078449 5904 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078452 5904 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078455 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078458 5904 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078460 5904 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078463 5904 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078466 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:29:13.082155 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078469 5904 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078471 5904 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078474 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078477 5904 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078479 5904 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078483 5904 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078485 5904 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078488 5904 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078491 5904 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078494 5904 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078497 5904 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078499 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078502 5904 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078505 5904 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078508 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078511 5904 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078513 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078516 5904 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078518 5904 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078521 5904 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:29:13.082842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078523 5904 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078526 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078528 5904 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078531 5904 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078534 5904 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078537 5904 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078540 5904 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078543 5904 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078545 5904 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078548 5904 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078551 5904 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078553 5904 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078556 5904 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078558 5904 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078561 5904 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078563 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078566 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078569 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078571 5904 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078574 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:29:13.083640 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078576 5904 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078579 5904 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078582 5904 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078584 5904 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078587 5904 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078589 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078592 5904 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078595 5904 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078597 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078600 5904 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078602 5904 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078605 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078608 5904 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078611 5904 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078613 5904 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078616 5904 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078619 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078622 5904 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078624 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:29:13.084262 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078627 5904 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078631 5904 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.078634 5904 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078731 5904 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078746 5904 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078752 5904 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078756 5904 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078760 5904 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078765 5904 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078769 5904 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078774 5904 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078778 5904 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078781 5904 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078784 5904 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078788 5904 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078791 5904 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078795 5904 flags.go:64] FLAG: --cgroup-root="" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078798 5904 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078801 5904 flags.go:64] FLAG: --client-ca-file="" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078804 5904 flags.go:64] FLAG: --cloud-provider="external" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078807 5904 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078812 5904 flags.go:64] FLAG: --cluster-domain="" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078815 5904 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 05:29:13.084753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078818 5904 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078822 5904 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078825 5904 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078830 5904 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078833 5904 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078836 5904 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078842 5904 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078846 5904 flags.go:64] FLAG: --contention-profiling="false" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078849 5904 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078852 5904 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078855 5904 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078858 5904 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078863 5904 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078866 5904 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078869 5904 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078872 5904 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078875 5904 flags.go:64] FLAG: --enable-server="true" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078879 5904 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078883 5904 flags.go:64] FLAG: --event-burst="100" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078886 5904 flags.go:64] FLAG: --event-qps="50" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078889 5904 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078892 5904 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078895 5904 flags.go:64] FLAG: --eviction-hard="" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078899 5904 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 05:29:13.085293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078903 5904 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078906 5904 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078913 5904 flags.go:64] FLAG: --eviction-soft="" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078917 5904 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078920 5904 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078923 5904 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078927 5904 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078930 5904 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078933 5904 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078936 5904 flags.go:64] FLAG: --feature-gates="" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078940 5904 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078943 5904 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078946 5904 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078949 5904 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078954 5904 flags.go:64] FLAG: --healthz-port="10248" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078958 5904 flags.go:64] FLAG: --help="false" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078961 5904 flags.go:64] FLAG: --hostname-override="ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078964 5904 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078968 5904 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078971 5904 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078974 5904 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078978 5904 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078981 5904 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 05:29:13.085882 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078984 5904 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078987 5904 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078990 5904 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078993 5904 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.078996 5904 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079000 5904 flags.go:64] FLAG: --kube-reserved="" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079003 5904 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079006 5904 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079009 5904 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079012 5904 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079015 5904 flags.go:64] FLAG: --lock-file="" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079018 5904 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079022 5904 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079025 5904 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079030 5904 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079033 5904 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079037 5904 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079040 5904 flags.go:64] FLAG: --logging-format="text" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079043 5904 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079047 5904 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079049 5904 flags.go:64] FLAG: --manifest-url="" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079052 5904 flags.go:64] FLAG: --manifest-url-header="" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079057 5904 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079061 5904 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079065 5904 flags.go:64] FLAG: --max-pods="110" Nov 04 05:29:13.086448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079068 5904 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079071 5904 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079075 5904 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079078 5904 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079081 5904 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079084 5904 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079087 5904 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079095 5904 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079098 5904 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079102 5904 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079105 5904 flags.go:64] FLAG: --pod-cidr="" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079108 5904 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079113 5904 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079116 5904 flags.go:64] FLAG: --pods-per-core="0" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079119 5904 flags.go:64] FLAG: --port="10250" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079123 5904 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079126 5904 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-01a4a5a33d44c4be0" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079129 5904 flags.go:64] FLAG: --qos-reserved="" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079132 5904 flags.go:64] FLAG: --read-only-port="10255" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079135 5904 flags.go:64] FLAG: --register-node="true" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079138 5904 flags.go:64] FLAG: --register-with-taints="" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079142 5904 flags.go:64] FLAG: --registry-burst="10" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079146 5904 flags.go:64] FLAG: --registry-qps="5" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079149 5904 flags.go:64] FLAG: --reserved-cpus="" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079152 5904 flags.go:64] FLAG: --reserved-memory="" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079155 5904 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 05:29:13.087043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079159 5904 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079162 5904 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079165 5904 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079168 5904 flags.go:64] FLAG: --runonce="false" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079172 5904 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079175 5904 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079178 5904 flags.go:64] FLAG: --seccomp-default="false" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079181 5904 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079184 5904 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079188 5904 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079191 5904 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079194 5904 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079197 5904 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079200 5904 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079204 5904 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079206 5904 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079210 5904 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079213 5904 flags.go:64] FLAG: --system-cgroups="" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079217 5904 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079237 5904 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079241 5904 flags.go:64] FLAG: --tls-cert-file="" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079244 5904 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079248 5904 flags.go:64] FLAG: --tls-min-version="" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079251 5904 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079254 5904 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 05:29:13.087826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079257 5904 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079260 5904 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079264 5904 flags.go:64] FLAG: --v="2" Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079268 5904 flags.go:64] FLAG: --version="false" Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079272 5904 flags.go:64] FLAG: --vmodule="" Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079276 5904 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079280 5904 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079411 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079414 5904 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079418 5904 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079421 5904 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079425 5904 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079428 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079431 5904 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079434 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079436 5904 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079439 5904 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079442 5904 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079444 5904 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079447 5904 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079450 5904 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:29:13.088461 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079452 5904 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079455 5904 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079458 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079460 5904 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079464 5904 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079467 5904 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079469 5904 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079472 5904 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079475 5904 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079477 5904 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079480 5904 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079483 5904 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079485 5904 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079488 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079491 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079494 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079497 5904 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079499 5904 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079502 5904 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079505 5904 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:29:13.088956 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079508 5904 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079510 5904 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079514 5904 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079517 5904 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079520 5904 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079522 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079525 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079527 5904 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079530 5904 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079533 5904 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079535 5904 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079538 5904 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079540 5904 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079543 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079545 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079548 5904 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079552 5904 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079554 5904 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079557 5904 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079560 5904 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:29:13.089654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079563 5904 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079565 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079568 5904 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079571 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079573 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079576 5904 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079578 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079581 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079584 5904 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079587 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079589 5904 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079592 5904 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079595 5904 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079597 5904 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079601 5904 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079605 5904 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079609 5904 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079613 5904 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079617 5904 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:29:13.090245 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079620 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079622 5904 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079625 5904 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079628 5904 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079631 5904 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079634 5904 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079636 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079639 5904 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079642 5904 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079645 5904 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079648 5904 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079651 5904 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079654 5904 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079657 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079661 5904 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079664 5904 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079667 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079670 5904 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079673 5904 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:29:13.090798 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079675 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:29:13.091452 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079678 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:29:13.091452 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079681 5904 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:29:13.091452 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079684 5904 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:29:13.091452 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079686 5904 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:29:13.091452 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079689 5904 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:29:13.091452 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079691 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:29:13.091452 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079694 5904 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:29:13.091452 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079698 5904 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:29:13.091452 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079701 5904 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:29:13.091452 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079703 5904 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:29:13.091452 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079706 5904 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:29:13.091452 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079708 5904 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:29:13.091452 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079711 5904 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:29:13.091452 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.079736 5904 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:29:13.091452 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079773 5904 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:29:13.091452 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079778 5904 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079781 5904 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079783 5904 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079786 5904 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079789 5904 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079792 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079794 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079798 5904 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079800 5904 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079803 5904 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079806 5904 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079808 5904 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079811 5904 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079814 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079816 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079819 5904 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079821 5904 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079824 5904 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079827 5904 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079830 5904 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:29:13.092146 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079832 5904 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079835 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079838 5904 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079840 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079844 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079847 5904 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079849 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079852 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079854 5904 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079857 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079859 5904 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079862 5904 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079865 5904 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079868 5904 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079871 5904 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079874 5904 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079878 5904 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079881 5904 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079884 5904 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:29:13.093067 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079888 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079891 5904 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079894 5904 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079896 5904 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079899 5904 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079902 5904 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079904 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079907 5904 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079910 5904 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079912 5904 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079915 5904 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079917 5904 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079920 5904 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079923 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079928 5904 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079932 5904 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079936 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079940 5904 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079943 5904 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:29:13.093744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079946 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079949 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079952 5904 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079955 5904 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079958 5904 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079960 5904 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079963 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079966 5904 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079968 5904 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079971 5904 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079973 5904 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079976 5904 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079979 5904 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079983 5904 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079985 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079988 5904 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079991 5904 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079993 5904 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079996 5904 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.079999 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:29:13.094293 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080001 5904 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080004 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080007 5904 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080009 5904 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080012 5904 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080015 5904 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080018 5904 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080021 5904 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080023 5904 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080026 5904 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080029 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080032 5904 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080035 5904 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080037 5904 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080040 5904 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080042 5904 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080045 5904 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080047 5904 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080051 5904 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:29:13.094775 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080053 5904 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080056 5904 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080059 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080061 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080064 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080066 5904 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080070 5904 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.080073 5904 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.085027 5904 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.085045 5904 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.085064 5904 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.085069 5904 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085125 5904 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085131 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085134 5904 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085137 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085139 5904 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085142 5904 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085145 5904 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085149 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:29:13.095269 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085152 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085155 5904 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085158 5904 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085161 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085164 5904 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085167 5904 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085170 5904 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085178 5904 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085181 5904 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085183 5904 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085186 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085189 5904 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085191 5904 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085194 5904 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085197 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085200 5904 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085203 5904 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085205 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085208 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085211 5904 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:29:13.095777 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085214 5904 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085217 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085219 5904 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085240 5904 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085246 5904 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085250 5904 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085254 5904 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085257 5904 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085260 5904 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085263 5904 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085266 5904 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085268 5904 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085271 5904 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085274 5904 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085276 5904 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085279 5904 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085282 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085285 5904 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085288 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:29:13.096440 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085291 5904 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085294 5904 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085297 5904 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085302 5904 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085306 5904 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085309 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085312 5904 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085315 5904 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085317 5904 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085320 5904 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085323 5904 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085326 5904 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085329 5904 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085331 5904 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085334 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085336 5904 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085339 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085342 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085345 5904 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:29:13.096924 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085348 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085350 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085354 5904 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085356 5904 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085359 5904 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085362 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085364 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085367 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085370 5904 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085372 5904 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085375 5904 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085378 5904 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085380 5904 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085383 5904 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085386 5904 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085389 5904 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085391 5904 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085394 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085397 5904 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085400 5904 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:29:13.097386 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085402 5904 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085405 5904 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085408 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085410 5904 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085414 5904 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085416 5904 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085419 5904 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085422 5904 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085425 5904 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085428 5904 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085430 5904 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085433 5904 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085436 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085438 5904 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085441 5904 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085444 5904 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085446 5904 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085449 5904 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085452 5904 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:29:13.097848 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085454 5904 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:29:13.098308 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.085483 5904 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:29:13.098308 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085626 5904 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:29:13.098308 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085630 5904 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:29:13.098308 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085633 5904 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:29:13.098308 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085636 5904 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:29:13.098308 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085639 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:29:13.098308 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085643 5904 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:29:13.098308 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085646 5904 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:29:13.098308 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085648 5904 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:29:13.098308 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085651 5904 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:29:13.098308 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085654 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:29:13.098308 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085657 5904 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:29:13.098308 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085659 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:29:13.098308 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085662 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:29:13.098308 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085665 5904 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:29:13.098308 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085668 5904 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085671 5904 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085674 5904 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085676 5904 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085679 5904 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085682 5904 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085685 5904 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085687 5904 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085690 5904 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085693 5904 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085695 5904 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085699 5904 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085703 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085706 5904 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085708 5904 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085711 5904 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085714 5904 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085716 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085719 5904 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085722 5904 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:29:13.098761 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085725 5904 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085727 5904 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085730 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085732 5904 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085736 5904 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085739 5904 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085741 5904 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085744 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085746 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085750 5904 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085755 5904 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085758 5904 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085761 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085764 5904 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085766 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085769 5904 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085772 5904 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085774 5904 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085777 5904 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:29:13.099658 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085780 5904 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085783 5904 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085786 5904 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085788 5904 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085791 5904 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085793 5904 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085796 5904 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085799 5904 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085801 5904 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085804 5904 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085806 5904 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085809 5904 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085812 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085814 5904 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085817 5904 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085819 5904 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085822 5904 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085825 5904 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085828 5904 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085832 5904 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:29:13.100138 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085834 5904 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085837 5904 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085839 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085843 5904 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085845 5904 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085848 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085851 5904 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085854 5904 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085856 5904 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085859 5904 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085862 5904 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085865 5904 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085867 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085870 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085873 5904 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085875 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085878 5904 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085881 5904 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085884 5904 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085887 5904 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:29:13.100642 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085889 5904 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:29:13.101159 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085892 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:29:13.101159 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085895 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:29:13.101159 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085898 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:29:13.101159 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085901 5904 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:29:13.101159 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085904 5904 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:29:13.101159 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085907 5904 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:29:13.101159 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085909 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:29:13.101159 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085912 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:29:13.101159 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085915 5904 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:29:13.101159 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085917 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:29:13.101159 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085923 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:29:13.101159 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085926 5904 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:29:13.101159 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.085949 5904 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:29:13.101159 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085981 5904 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:29:13.101159 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085985 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:29:13.101159 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085988 5904 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085991 5904 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085994 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.085998 5904 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086000 5904 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086004 5904 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086006 5904 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086009 5904 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086012 5904 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086015 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086018 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086021 5904 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086024 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086027 5904 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086030 5904 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086033 5904 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086035 5904 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086038 5904 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086041 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086045 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:29:13.101629 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086047 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086050 5904 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086053 5904 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086055 5904 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086058 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086061 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086063 5904 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086066 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086069 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086072 5904 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086074 5904 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086077 5904 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086080 5904 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086082 5904 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086085 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086088 5904 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086090 5904 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086093 5904 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086096 5904 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:29:13.102108 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086099 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086101 5904 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086104 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086106 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086109 5904 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086112 5904 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086115 5904 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086117 5904 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086120 5904 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086123 5904 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086125 5904 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086128 5904 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086131 5904 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086133 5904 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086136 5904 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086139 5904 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086142 5904 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086146 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086149 5904 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086152 5904 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:29:13.102618 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086155 5904 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086158 5904 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086161 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086164 5904 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086167 5904 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086169 5904 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086172 5904 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086175 5904 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086180 5904 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086183 5904 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086186 5904 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086188 5904 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086191 5904 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086194 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086198 5904 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086202 5904 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086205 5904 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086209 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086212 5904 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:29:13.103144 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086215 5904 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086218 5904 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086221 5904 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086260 5904 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086265 5904 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086268 5904 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086270 5904 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086273 5904 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086276 5904 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086279 5904 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086282 5904 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086284 5904 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086287 5904 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086290 5904 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086292 5904 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086300 5904 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086304 5904 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086307 5904 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086310 5904 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086312 5904 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:29:13.103654 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086315 5904 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:29:13.104167 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086321 5904 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:29:13.104167 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086323 5904 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:29:13.104167 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086326 5904 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:29:13.104167 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086331 5904 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:29:13.104167 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:13.086333 5904 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:29:13.104167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.086392 5904 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 05:29:13.104167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.087372 5904 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Nov 04 05:29:13.104167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.087425 5904 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Nov 04 05:29:13.104167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.087609 5904 server.go:1014] "Starting client certificate rotation" Nov 04 05:29:13.104167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.087752 5904 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 05:29:13.104167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.087799 5904 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 05:23:03 +0000 UTC" deadline="2028-04-02 15:56:42.037219287 +0000 UTC" Nov 04 05:29:13.104167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.087827 5904 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12370h27m28.949394722s" Nov 04 05:29:13.104167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.088040 5904 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 05:29:13.104167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.089075 5904 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 05:29:13.104167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.090715 5904 log.go:25] "Validated CRI v1 runtime API" Nov 04 05:29:13.104167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.093169 5904 log.go:25] "Validated CRI v1 image API" Nov 04 05:29:13.104572 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.093915 5904 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 05:29:13.104572 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.096319 5904 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8d969107-3d79-4478-b37d-8eaec9420231:/dev/nvme0n1p4 c6586ec7-5431-4d80-92d2-d95ded28bf70:/dev/nvme0n1p3] Nov 04 05:29:13.104666 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.096335 5904 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/150dd6af1f9754f9d50d8336979b00a0ab8946403e61290e66a426bc4fcf64a4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/150dd6af1f9754f9d50d8336979b00a0ab8946403e61290e66a426bc4fcf64a4/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2d864e2af4c097fae3a6adc8a49ec7a6bf772854923fd38a9dd162cf84666664/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2d864e2af4c097fae3a6adc8a49ec7a6bf772854923fd38a9dd162cf84666664/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/37daf502ec6a620564fe64fcf8e755709761e5b543b611a8c11c693e07680569/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/37daf502ec6a620564fe64fcf8e755709761e5b543b611a8c11c693e07680569/userdata/shm major:0 minor:203 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3f5185ad9371ea13cbae73e1eb02bfa3a76d165dcc331d716aca0b8b2ffdcc12/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3f5185ad9371ea13cbae73e1eb02bfa3a76d165dcc331d716aca0b8b2ffdcc12/userdata/shm major:0 minor:231 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4a065ef3d59ba12dd48cf4231807c8c00afb5d135f136d7dabfe7b53b2356aea/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4a065ef3d59ba12dd48cf4231807c8c00afb5d135f136d7dabfe7b53b2356aea/userdata/shm major:0 minor:337 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/54e8b758848496b767942024732e9c6aa18697a874cd125465400a2159011c38/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/54e8b758848496b767942024732e9c6aa18697a874cd125465400a2159011c38/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/579c3d33f20c142fe1026324599c72f7c34a4f83fd3677b35a2ececb10451dba/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/579c3d33f20c142fe1026324599c72f7c34a4f83fd3677b35a2ececb10451dba/userdata/shm major:0 minor:268 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/68869ffb949af724efeba8265deadbac63f5df90c9be986b392d98c3e12d09c1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/68869ffb949af724efeba8265deadbac63f5df90c9be986b392d98c3e12d09c1/userdata/shm major:0 minor:389 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/69524c5d81dbedb21b8902882d183dd54090d0c8bba09f1bb720201a28cc478e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/69524c5d81dbedb21b8902882d183dd54090d0c8bba09f1bb720201a28cc478e/userdata/shm major:0 minor:505 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/75fd2d220aed24cc4be18d597bc21fe0ae142ab309de07a195021252a7217aa6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/75fd2d220aed24cc4be18d597bc21fe0ae142ab309de07a195021252a7217aa6/userdata/shm major:0 minor:452 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/81d3b7d850ec27e11ee9ad7f1af5b6c3f78fd747b809dddf7099656a34dbed8f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/81d3b7d850ec27e11ee9ad7f1af5b6c3f78fd747b809dddf7099656a34dbed8f/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/88caa64b9200e78dbe39b56806babe65cda107d9b4711b1f4010c8c959b530b9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/88caa64b9200e78dbe39b56806babe65cda107d9b4711b1f4010c8c959b530b9/userdata/shm major:0 minor:71 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/88dbe066e70488397c0c8a44b49766afe9dd1661739d3a6f1b29d418b982ea80/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/88dbe066e70488397c0c8a44b49766afe9dd1661739d3a6f1b29d418b982ea80/userdata/shm major:0 minor:423 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/90c9151a2fd867fa92e53aa4a1598e337ebe1bb24b81be305a6c9b7db4fa2aef/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/90c9151a2fd867fa92e53aa4a1598e337ebe1bb24b81be305a6c9b7db4fa2aef/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/94135939bff49a0fa311a9f073dc3229f16aed854b2416ea53fd78eab0f2adb6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/94135939bff49a0fa311a9f073dc3229f16aed854b2416ea53fd78eab0f2adb6/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9c8898951746b209c4c5d3ff9c217388d6a6b5bbbed8bb14d7ff0e46c6c609be/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9c8898951746b209c4c5d3ff9c217388d6a6b5bbbed8bb14d7ff0e46c6c609be/userdata/shm major:0 minor:299 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/af6995ea37d42bd2c4eb3ba220696c0b46e67491722cdb5ebbdbde5bad41cfb6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/af6995ea37d42bd2c4eb3ba220696c0b46e67491722cdb5ebbdbde5bad41cfb6/userdata/shm major:0 minor:504 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bd55eee618c292174656ff666436e101d1d4a8d706c27c8dfc320f56309eed53/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bd55eee618c292174656ff666436e101d1d4a8d706c27c8dfc320f56309eed53/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ca79ba46ecc6a6f60d1cc0136524237031a249b5f03a020926b52a6d7576b16d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ca79ba46ecc6a6f60d1cc0136524237031a249b5f03a020926b52a6d7576b16d/userdata/shm major:0 minor:367 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cc61bb02f57f0ef6474fb326691082adb0a9cfd869f7df4548c847266c2dfa14/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cc61bb02f57f0ef6474fb326691082adb0a9cfd869f7df4548c847266c2dfa14/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d96ea9b49f192b58243c43e8fafd06a4fd251b6290ffc75d0a5439abbe487bff/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d96ea9b49f192b58243c43e8fafd06a4fd251b6290ffc75d0a5439abbe487bff/userdata/shm major:0 minor:502 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/de37f3c4cee214cc69bffc454ecda72e0f9142c2af060ccde47e2e192b378ac7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/de37f3c4cee214cc69bffc454ecda72e0f9142c2af060ccde47e2e192b378ac7/userdata/shm major:0 minor:272 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f1621b0505a7ee1a889b867df2295e11c7af927f0b50f6eaab9619290dcdfd65/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f1621b0505a7ee1a889b867df2295e11c7af927f0b50f6eaab9619290dcdfd65/userdata/shm major:0 minor:262 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f5e1af278e75b5d52d7966e63de05e330aeb88b38c16ff21c0f46ccff5bf9732/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f5e1af278e75b5d52d7966e63de05e330aeb88b38c16ff21c0f46ccff5bf9732/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fce1cb0777caf725040593cf1a7448767e27710547736781c8687537530dd25d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fce1cb0777caf725040593cf1a7448767e27710547736781c8687537530dd25d/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ffc140d9d862cd089d5b3cd81b402ab2707df60ab6400c5a38fef3123c5b7074/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ffc140d9d862cd089d5b3cd81b402ab2707df60ab6400c5a38fef3123c5b7074/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/137f70c1-c852-4c5f-823f-b8f7ed565c00/volumes/kubernetes.io~projected/kube-api-access-c76c8:{mountpoint:/var/lib/kubelet/pods/137f70c1-c852-4c5f-823f-b8f7ed565c00/volumes/kubernetes.io~projected/kube-api-access-c76c8 major:0 minor:260 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/137f70c1-c852-4c5f-823f-b8f7ed565c00/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/137f70c1-c852-4c5f-823f-b8f7ed565c00/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:253 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/137f70c1-c852-4c5f-823f-b8f7ed565c00/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/137f70c1-c852-4c5f-823f-b8f7ed565c00/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:256 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/32bea234-f8f1-41fa-ac2f-bea97c729d10/volumes/kubernetes.io~projected/kube-api-access-mrnng:{mountpoint:/var/lib/kubelet/pods/32bea234-f8f1-41fa-ac2f-bea97c729d10/volumes/kubernetes.io~projected/kube-api-access-mrnng major:0 minor:362 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/32bea234-f8f1-41fa-ac2f-bea97c729d10/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/32bea234-f8f1-41fa-ac2f-bea97c729d10/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:360 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/32bea234-f8f1-41fa-ac2f-bea97c729d10/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/32bea234-f8f1-41fa-ac2f-bea97c729d10/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:359 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/32bea234-f8f1-41fa-ac2f-bea97c729d10/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/32bea234-f8f1-41fa-ac2f-bea97c729d10/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:361 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/343f8865-3015-4715-960c-a2e1418c6f7e/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/343f8865-3015-4715-960c-a2e1418c6f7e/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/343f8865-3015-4715-960c-a2e1418c6f7e/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/343f8865-3015-4715-960c-a2e1418c6f7e/volumes/kubernetes.io~empty-dir/tmp major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/343f8865-3015-4715-960c-a2e1418c6f7e/volumes/kubernetes.io~projected/kube-api-access-x6rf9:{mountpoint:/var/lib/kubelet/pods/343f8865-3015-4715-960c-a2e1418c6f7e/volumes/kubernetes.io~projected/kube-api-access-x6rf9 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/379bdc95-36d7-47f2-9e17-a96e51da6495/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/379bdc95-36d7-47f2-9e17-a96e51da6495/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:501 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/47be26f2-143e-4b49-824a-415e60f6140b/volumes/kubernetes.io~projected/kube-api-access-h4b8q:{mountpoint:/var/lib/kubelet/pods/47be26f2-143e-4b49-824a-415e60f6140b/volumes/kubernetes.io~projected/kube-api-access-h4b8q major:0 minor:500 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01/volumes/kubernetes.io~projected/kube-api-access-5gztg:{mountpoint:/var/lib/kubelet/pods/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01/volumes/kubernetes.io~projected/kube-api-access-5gztg major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52797b3b-80b4-443e-aa1d-158430ee739e/volumes/kubernetes.io~projected/kube-api-access-sjz6h:{mountpoint:/var/lib/kubelet/pods/52797b3b-80b4-443e-aa1d-158430ee739e/volumes/kubernetes.io~projected/kube-api-access-sjz6h major:0 minor:374 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52797b3b-80b4-443e-aa1d-158430ee739e/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/52797b3b-80b4-443e-aa1d-158430ee739e/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:370 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7239836a-64ce-4847-91f4-9360cd652ea6/volumes/kubernetes.io~projected/kube-api-access-pzqnm:{mountpoint:/var/lib/kubelet/pods/7239836a-64ce-4847-91f4-9360cd652ea6/volumes/kubernetes.io~projected/kube-api-access-pzqnm major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7374bdb0-a62f-4b1c-9d14-e1097c851ff0/volumes/kubernetes.io~projected/kube-api-access-wlrrj:{mountpoint:/var/lib/kubelet/pods/7374bdb0-a62f-4b1c-9d14-e1097c851ff0/volumes/kubernetes.io~projected/kube-api-access-wlrrj major:0 minor:261 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7374bdb0-a62f-4b1c-9d14-e1097c851ff0/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/7374bdb0-a62f-4b1c-9d14-e1097c851ff0/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:257 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7374bdb0-a62f-4b1c-9d14-e1097c851ff0/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/7374bdb0-a62f-4b1c-9d14-e1097c851ff0/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:255 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7beaf096-fa6b-4233-a68b-1ab974c2e037/volumes/kubernetes.io~projected/kube-api-access-xnhgf:{mountpoint:/var/lib/kubelet/pods/7beaf096-fa6b-4233-a68b-1ab974c2e037/volumes/kubernetes.io~projected/kube-api-access-xnhgf major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7beaf096-fa6b-4233-a68b-1ab974c2e037/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/7beaf096-fa6b-4233-a68b-1ab974c2e037/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7beaf096-fa6b-4233-a68b-1ab974c2e037/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/7beaf096-fa6b-4233-a68b-1ab974c2e037/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/80902534-76da-4e9d-8157-ce481e71f2f8/volumes/kubernetes.io~projected/kube-api-access-ghglb:{mountpoint:/var/lib/kubelet/pods/80902534-76da-4e9d-8157-ce481e71f2f8/volumes/kubernetes.io~projected/kube-api-access-ghglb major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/82660df8-f278-4d7c-b02a-149527e30d27/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/82660df8-f278-4d7c-b02a-149527e30d27/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/82660df8-f278-4d7c-b02a-149527e30d27/volumes/kubernetes.io~projected/kube-api-access-pjjl9:{mountpoint:/var/lib/kubelet/pods/82660df8-f278-4d7c-b02a-149527e30d27/volumes/kubernetes.io~projected/kube-api-access-pjjl9 major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/82660df8-f278-4d7c-b02a-149527e30d27/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/82660df8-f278-4d7c-b02a-149527e30d27/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/915dd778-3692-4f41-83d3-cc5d4e73d615/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/915dd778-3692-4f41-83d3-cc5d4e73d615/volumes/kubernetes.io~secret/agent-certs major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9f075074-0903-4de4-b247-59a26f5b5f95/volumes/kubernetes.io~projected/kube-api-access-mt7bc:{mountpoint:/var/lib/kubelet/pods/9f075074-0903-4de4-b247-59a26f5b5f95/volumes/kubernetes.io~projected/kube-api-access-mt7bc major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9f075074-0903-4de4-b247-59a26f5b5f95/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/9f075074-0903-4de4-b247-59a26f5b5f95/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c93278df-81cb-480e-9151-ba3056f43d74/volumes/kubernetes.io~projected/kube-api-access-wzdc9:{mountpoint:/var/lib/kubelet/pods/c93278df-81cb-480e-9151-ba3056f43d74/volumes/kubernetes.io~projected/kube-api-access-wzdc9 major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c93278df-81cb-480e-9151-ba3056f43d74/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/c93278df-81cb-480e-9151-ba3056f43d74/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202/volumes/kubernetes.io~projected/kube-api-access-rc46j:{mountpoint:/var/lib/kubelet/pods/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202/volumes/kubernetes.io~projected/kube-api-access-rc46j major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volume-subpaths/web-config/alertmanager/9 major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volume-subpaths/web-config/config-reloader/6 major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~empty-dir/config-out major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~projected/kube-api-access-28fm9:{mountpoint:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~projected/kube-api-access-28fm9 major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~projected/tls-assets major:0 minor:298 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/config-volume major:0 minor:292 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:291 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/web-config major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15/volumes/kubernetes.io~projected/kube-api-access-ddw4d:{mountpoint:/var/lib/kubelet/pods/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15/volumes/kubernetes.io~projected/kube-api-access-ddw4d major:0 minor:259 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:254 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:258 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e19ce810-1d70-46db-a8ce-d824a9ffc81e/volumes/kubernetes.io~projected/kube-api-access-x8wjt:{mountpoint:/var/lib/kubelet/pods/e19ce810-1d70-46db-a8ce-d824a9ffc81e/volumes/kubernetes.io~projected/kube-api-access-x8wjt major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e19ce810-1d70-46db-a8ce-d824a9ffc81e/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/e19ce810-1d70-46db-a8ce-d824a9ffc81e/volumes/kubernetes.io~secret/metrics-tls major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~projected/kube-api-access-9dmt9:{mountpoint:/var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~projected/kube-api-access-9dmt9 major:0 minor:347 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:346 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:341 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:345 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:344 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:343 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:342 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e39328c2-e756-4c99-9d13-6fbdccae91cb/volumes/kubernetes.io~projected/kube-api-access-5mwlv:{mountpoint:/var/lib/kubelet/pods/e39328c2-e756-4c99-9d13-6fbdccae91cb/volumes/kubernetes.io~projected/kube-api-access-5mwlv major:0 minor:409 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e39328c2-e756-4c99-9d13-6fbdccae91cb/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/e39328c2-e756-4c99-9d13-6fbdccae91cb/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:413 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e39328c2-e756-4c99-9d13-6fbdccae91cb/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/e39328c2-e756-4c99-9d13-6fbdccae91cb/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:414 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volume-subpaths/web-config/config-reloader/2 major:0 minor:449 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:449 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volume-subpaths/web-config/prometheus/15 major:0 minor:449 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~empty-dir/config-out major:0 minor:444 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~projected/kube-api-access-d92sv:{mountpoint:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~projected/kube-api-access-d92sv major:0 minor:450 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~projected/tls-assets major:0 minor:448 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/config major:0 minor:451 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:443 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:446 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:447 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:441 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:445 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:442 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:436 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/web-config major:0 minor:449 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/efbcf64e-3ade-4343-a9c8-bf945b141a02/volumes/kubernetes.io~projected/kube-api-access-kg65k:{mountpoint:/var/lib/kubelet/pods/efbcf64e-3ade-4343-a9c8-bf945b141a02/volumes/kubernetes.io~projected/kube-api-access-kg65k major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/efbcf64e-3ade-4343-a9c8-bf945b141a02/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/efbcf64e-3ade-4343-a9c8-bf945b141a02/volumes/kubernetes.io~secret/metrics-certs major:0 minor:496 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f4a51628-0aab-4614-a4ed-11f184521cd4/volumes/kubernetes.io~projected/kube-api-access-nvb55:{mountpoint:/var/lib/kubelet/pods/f4a51628-0aab-4614-a4ed-11f184521cd4/volumes/kubernetes.io~projected/kube-api-access-nvb55 major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7b4c728-a80c-403d-88d9-af4fb0835993/volumes/kubernetes.io~projected/kube-api-access-tc4df:{mountpoint:/var/lib/kubelet/pods/f7b4c728-a80c-403d-88d9-af4fb0835993/volumes/kubernetes.io~projected/kube-api-access-tc4df major:0 minor:48 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/3703555e7eb214e5e762a704f2d02cc64d06797f3b0271a4ed29a8580f77a2ef/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/0f08b19a088b39f99f8579d660b54a2befc1e7941e0e656ac38c2686e9f2fa21/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/d6d709d7fbb6c4ffe8a2b5a51db26ddbfc1a29c14d2198045c9609663d34fd36/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/6e88bd578006b2e7fb7e1bbec7abc9caa1aecab88a21eddebefa61d46b06dc62/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-111:{mountpoint:/var/lib/containers/storage/overlay/55c402272965fcb883f24278d2cd7c870468652735744d8c46ad2deece101aae/merged major:0 minor:111 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/d76bdeddcf243ca72837d3fbe9d87ffc4d6700e518e904cc5c4184a030d2d9b8/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/33a58814534e20cca2baf58d8098bee6111b7111099098e8c752273e0a39fdcb/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/a2b87a9f3b87fbc226c232e66f76af9dd5ce366effa7470d4d3d8272bd03b8ef/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/00b34a80179d86d552762e7140e1669cd1c6a1dbf2b56e32eb092d8b18886c95/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/611dfe8c8634f448be9cb5a401b5358b250dfb99a2884c731595ae6599a4f528/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/4ffdb1e072af3c44a4415c2f3bdcafecd4d77da8b77d1bfb9f898fd789472c11/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-148:{mountpoint:/var/lib/containers/storage/overlay/cc63c5427b8ceab366e8391e25cc42173393683962ce3d329bb2a633b0c7426d/merged major:0 minor:148 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/a166d052a420f64a101df442e34d2860e36ef376c870ab510ab56d1826654dbd/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/3bd7d8997cd8e9bb6fa99081527523dd33c42b9ec1fe7baa5fb3658233c67156/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/5ecca0c59ff2bf9116cf4bdcf7e5347301a71e2079a45f56c512ee09f4cc47e1/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/6622e24f4234aa598495f3af4a64b51fc8305d1c743be85763be92e66714c1f8/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/8fac7adf7f4aa52f5cf8f1593a05e1bef75c2a1838f0fac2b30ca62635be167e/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/d3a7b37b775ef9d1a70847638a9a7fdc161eae98b6347b802e9458d4f4674ab2/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/dfecbe3b64195f49725a1eb7e84d57b443fec768b089da6ebe6c56ced232e37b/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/92a685606d096be983896bc20f76c627a6232fee8937b89916f8e00213616394/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/321b1e2e295a8f82196bcbfbb851cd45ffb2f5bccf4e836ab0fda6e20fb0973f/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/2c560b176ec52a3967c749ba32fc847f3bce07c34bcbe5a74db3071ac86d969c/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/3a229a9ad41db3740934272dade315d31ba8a57af9dd9940329043df3e59a2df/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/8d1d3265ef5f88bb2738a12adb4ccc8944dff118d5dbe1cff4d5dc0722950e13/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/5f6027df1fe586e59df6187b4de299c6de7ea67717c156fd3c4f5cab76b29ef2/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/44ef44b23b2044f9b86a992b280f20555601ab7b2a9f2d9f1ef0ceb0409d83a8/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-233:{mountpoint:/var/lib/containers/storage/overlay/91e39e82487514d53cc81aa8bf78102f9f5a9fcb564547d258dd684bcea04424/merged major:0 minor:233 fsType:overlay blockSize:0} overlay_0-235:{mountpoint:/var/lib/containers/storage/overlay/408991e48ced9842b3b495212de35bd3bd45fb0cb912573a81158cec8244ae71/merged major:0 minor:235 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/76b43f6647c5d69d372f0b04736a780dd8de8c8e7d783313bf0d241ec6f41aee/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-243:{mountpoint:/var/lib/containers/storage/overlay/fe14ce235e9d396fc0cf00869cfd76e8f2866df14cb2d1ffd60b0b36696b0cdc/merged major:0 minor:243 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/905c85a143d95bec1bae13b3d53d478daf6e541b9aedf1246f168fac1b96d0ab/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/8a676ad4b09b489dd301951c3ab9d8c62c33e4ff1f212c3e747653fdbead7f92/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-274:{mountpoint:/var/lib/containers/storage/overlay/629e0dcceedfdcce0babbc04f2288328d7dc35dba620e7de7e79bb5390ccfd0c/merged major:0 minor:274 fsType:overlay blockSize:0} overlay_0-276:{mountpoint:/var/lib/containers/storage/overlay/86c296468772039b27fc73e9aebe542cb7d4606bdbcdab790efe4d1734ebae4b/merged major:0 minor:276 fsType:overlay blockSize:0} overlay_0-278:{mountpoint:/var/lib/containers/storage/overlay/8e4758a358b4ad600c0010743f206e7f98d39ff6899bd7849126491c4d0711f3/merged major:0 minor:278 fsType:overlay blockSize:0} overlay_0-279:{mountpoint:/var/lib/containers/storage/overlay/f0902305c84b7120e46b06c42b61e57fa33959a1b30c076e6a77212a6123b51f/merged major:0 minor:279 fsType:overlay blockSize:0} overlay_0-301:{mountpoint:/var/lib/containers/storage/overlay/7d216bcc0db3a914529db75aa9ce10a5d87f0f9823772cb105b379e152af6abb/merged major:0 minor:301 fsType:overlay blockSize:0} overlay_0-303:{mountpoint:/var/lib/containers/storage/overlay/2a54dad29be98b113e8acd216ad1673ddd7ba4aec1fa9cef08d2669bd2e5c0ae/merged major:0 minor:303 fsType:overlay blockSize:0} overlay_0-305:{mountpoint:/var/lib/containers/storage/overlay/8d5316b28be2e7507a36e0fc9b6ea240960633d2fda206c16b1846a103a48353/merged major:0 minor:305 fsType:overlay blockSize:0} overlay_0-307:{mountpoint:/var/lib/containers/storage/overlay/86a4f1730368d857ad8ece94bb3e1a554a96b7ade24161a5bcebfc2e97f52a2f/merged major:0 minor:307 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/6a2aa642111f6c077dee492ef2be7fb02a73b1d5bd4da8fb7750109a4db212ce/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-322:{mountpoint:/var/lib/containers/storage/overlay/da3b98fd522946e34d7c69987dbd8b81e27b142727ffffa63747c1e0629849c6/merged major:0 minor:322 fsType:overlay blockSize:0} overlay_0-327:{mountpoint:/var/lib/containers/storage/overlay/af56fb18a67c98472512ccf640fa2221d7df3e9a4b9369a2f007ff7378dfb24e/merged major:0 minor:327 fsType:overlay blockSize:0} overlay_0-329:{mountpoint:/var/lib/containers/storage/overlay/771ceb5deb5ff919344bec1d29c5cca6cb4d8650565bf2267efbafe474bdfbb2/merged major:0 minor:329 fsType:overlay blockSize:0} overlay_0-349:{mountpoint:/var/lib/containers/storage/overlay/2e238bfb39f980465830973e007ffe90ee5701f67962002d21fd4786f8907a90/merged major:0 minor:349 fsType:overlay blockSize:0} overlay_0-351:{mountpoint:/var/lib/containers/storage/overlay/39f386dd22f1fa27ce983175429fcb7f66203e3082304a2ca3c1cb4abc8a971b/merged major:0 minor:351 fsType:overlay blockSize:0} overlay_0-353:{mountpoint:/var/lib/containers/storage/overlay/5c0cf8781c797e8bec0a9cf8705ca9ac4e1dc17caef1ce86e021fbbd27630fca/merged major:0 minor:353 fsType:overlay blockSize:0} overlay_0-363:{mountpoint:/var/lib/containers/storage/overlay/4eefaf081e356f95b5215e50835082c46343a8768cd87c0eadf0da3206e2060d/merged major:0 minor:363 fsType:overlay blockSize:0} overlay_0-375:{mountpoint:/var/lib/containers/storage/overlay/cbe658b34532e85f2748cae0ca7c5ab375ec6eb5858e6705317eb10f8b78ffc4/merged major:0 minor:375 fsType:overlay blockSize:0} overlay_0-377:{mountpoint:/var/lib/containers/storage/overlay/926f21703fafc311afd7bf4f376f17ee7cca4d025ad3998b7856b44b8287ff19/merged major:0 minor:377 fsType:overlay blockSize:0} overlay_0-379:{mountpoint:/var/lib/containers/storage/overlay/188cb095b4edd7bf13127a168789fa5b1dc6ee7e278e8541ce94136cd9e01fd9/merged major:0 minor:379 fsType:overlay blockSize:0} overlay_0-385:{mountpoint:/var/lib/containers/storage/overlay/5fb06b50a09b654be6b2b796166c6dabf7eb311aad76b32699fa2ba4ffa41c6c/merged major:0 minor:385 fsType:overlay blockSize:0} overlay_0-392:{mountpoint:/var/lib/containers/storage/overlay/a3722dc2cab086879efc3d604a992a1fd801ca6d79c7315be111cc3761e72431/merged major:0 minor:392 fsType:overlay blockSize:0} overlay_0-397:{mountpoint:/var/lib/containers/storage/overlay/c7b49f5cd51228c1eeb014c43b2a09f8fc083ed66dee3a0a01e7dea02a27b695/merged major:0 minor:397 fsType:overlay blockSize:0} overlay_0-402:{mountpoint:/var/lib/containers/storage/overlay/bf3e2f2aa9c3e69fa59a9bcfbc0fc50eb2c83b0cc8f3906bf6196813514a5183/merged major:0 minor:402 fsType:overlay blockSize:0} overlay_0-407:{mountpoint:/var/lib/containers/storage/overlay/a75615aa323e25607fffbde82b65703175d44c0b9a0db10e1fc8ef7502239748/merged major:0 minor:407 fsType:overlay blockSize:0} overlay_0-415:{mountpoint:/var/lib/containers/storage/overlay/929bb863764c3bf7ceccd66b4b330f3df904f88c13180c0f6c297c4a04f90137/merged major:0 minor:415 fsType:overlay blockSize:0} overlay_0-421:{mountpoint:/var/lib/containers/storage/overlay/6d7d5aea1795085355d86ede56bab824a018c0ffa76fa0ef1f8f39fe1a44fced/merged major:0 minor:421 fsType:overlay blockSize:0} overlay_0-426:{mountpoint:/var/lib/containers/storage/overlay/49499e599db7797d925ecc338d6229c2384de3039361866d18afa62cdfa618f5/merged major:0 minor:426 fsType:overlay blockSize:0} overlay_0-432:{mountpoint:/var/lib/containers/storage/overlay/4862a0f8612e442b828e11241a30bbf0b3a7163006eae73f71706cee21a0da37/merged major:0 minor:432 fsType:overlay blockSize:0} overlay_0-434:{mountpoint:/var/lib/containers/storage/overlay/f3b74380b46fab1d650949d75b92b672a4c1ecbab69290aa9dd86ccb157bffd0/merged major:0 minor:434 fsType:overlay blockSize:0} overlay_0-454:{mountpoint:/var/lib/containers/storage/overlay/f08ee8b463ec3e3230ff03576cb7733e480d22e2c4e3b50316b7fd690f554ee3/merged major:0 minor:454 fsType:overlay blockSize:0} overlay_0-456:{mountpoint:/var/lib/containers/storage/overlay/26e1a34d171cfbf5238fb72e3f56f549dc670671b04f3e1b7e3ca5f1c42d5f07/merged major:0 minor:456 fsType:overlay blockSize:0} overlay_0-462:{mountpoint:/var/lib/containers/storage/overlay/309784cfe8b36928475ebfcd0b96982ca2a73b0f76ffc243190e196879e2566a/merged major:0 minor:462 fsType:overlay blockSize:0} overlay_0-464:{mountpoint:/var/lib/containers/storage/overlay/bb211bc2fc563c127adfc28c8fa96f14091cc7d0c97f928001efd27342fe41c6/merged major:0 minor:464 fsType:overlay blockSize:0} overlay_0-466:{mountpoint:/var/lib/containers/storage/overlay/88eb929ee082ffbfec9817435484d457c2cb2e48f9bd892d94287a5147882484/merged major:0 minor:466 fsType:overlay blockSize:0} overlay_0-476:{mountpoint:/var/lib/containers/storage/overlay/8d3043d6fcb8d72217074cbbb9f7ea72a1a3b2cf68847b240806f1672ebe14f2/merged major:0 minor:476 fsType:overlay blockSize:0} overlay_0-481:{mountpoint:/var/lib/containers/storage/overlay/baca34e50be6d50854a8c3df9d1d05e1d913a8d3f3a79e8f35793da039887fb7/merged major:0 minor:481 fsType:overlay blockSize:0} overlay_0-486:{mountpoint:/var/lib/containers/storage/overlay/05742d431ebae0ebc1838f0bf8cb34148c2c6c94cde732027150332b3e1454b9/merged major:0 minor:486 fsType:overlay blockSize:0} overlay_0-491:{mountpoint:/var/lib/containers/storage/overlay/787c29e9df8759735b6a15bb54542ff7f9a13afa5a1e944737688984244aeedd/merged major:0 minor:491 fsType:overlay blockSize:0} overlay_0-508:{mountpoint:/var/lib/containers/storage/overlay/945f486f552e6cba9dee8045feddd112c0a8aca0e45feccd873d3521c9953be3/merged major:0 minor:508 fsType:overlay blockSize:0} overlay_0-510:{mountpoint:/var/lib/containers/storage/overlay/9438e24934d42997ea2d3b1a297ee03b851d2c8a6f1c1d069ca8962b94a63cc5/merged major:0 minor:510 fsType:overlay blockSize:0} overlay_0-512:{mountpoint:/var/lib/containers/storage/overlay/057a880defe620b3728176e18cedeffeec3ce030127720a6d40d09f68b95d9c5/merged major:0 minor:512 fsType:overlay blockSize:0} overlay_0-514:{mountpoint:/var/lib/containers/storage/overlay/fc8a4366e8823b3ff8c2c519d5418b0dd152520abcf3da3179105856c3db4762/merged major:0 minor:514 fsType:overlay blockSize:0} overlay_0-516:{mountpoint:/var/lib/containers/storage/overlay/39de719ea9b95c085e368565579a4d32f6b48c67d628f92c27f30bb5bf5350a7/merged major:0 minor:516 fsType:overlay blockSize:0} overlay_0-522:{mountpoint:/var/lib/containers/storage/overlay/a079543cf4befb3ccbcf25ad99c5c0e58f6bf2097a32aa3c538e5d1c127dcefa/merged major:0 minor:522 fsType:overlay blockSize:0} overlay_0-527:{mountpoint:/var/lib/containers/storage/overlay/8fa5891295a2489531bacd143e86be490af8ce68b44e8e1a7a297a6cc28354b3/merged major:0 minor:527 fsType:overlay blockSize:0} overlay_0-64:{mountpoint:/var/lib/containers/storage/overlay/582f9a466db88eb5b84adc1a7b6a2c67d925f4a6e7b3a953a4614a071429ce5d/merged major:0 minor:64 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/7d15b26653effa3f401cd230a655528973945168e333f689da9dc254825c17a7/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/1646c1f089c03bf0313c66d4d2f0b345a765c492f7a8f76b41e7094e9452a86a/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/10c14f603226710e5748b8afe437a25404f65531c2353902e17c0e4a155a9fc1/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/190b18239f63741631fe5fc92fb404c3a1ca95af6a13195f9119577aee175bac/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/00a5994b018b5b7a74a04b53008c227e42b9047d1ee3fb699144a72f3de8ed8f/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/cc5780ea2b60fdffeaaf44547e68533e8b8b584f4c9d87f07670fa057e9da259/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/74021aeab05da291df2a6a649e1ae645ff07ce4791cde8756e5494a68df9e3bc/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/6e80fd7bdcce036950ad0f7bfd2c28e86a5e2250033b98bfcf60aa88bb899045/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/ad1bfefdb0a60ccbf6315da746594c584f099575bf8dc21c843da58f9efed6df/merged major:0 minor:96 fsType:overlay blockSize:0}] Nov 04 05:29:13.106504 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.106000 5904 manager.go:217] Machine: {Timestamp:2026-11-04 05:29:13.105503304 +0000 UTC m=+0.063088249 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3111595 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec224560ebdf1dc4588f1bbcd6e09ad9 SystemUUID:ec224560-ebdf-1dc4-588f-1bbcd6e09ad9 BootID:bf1c9f86-8385-409f-a364-d5ff5618fe4b Filesystems:[{Device:overlay_0-432 DeviceMajor:0 DeviceMinor:432 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-148 DeviceMajor:0 DeviceMinor:148 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9f075074-0903-4de4-b247-59a26f5b5f95/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:298 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:436 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-454 DeviceMajor:0 DeviceMinor:454 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-486 DeviceMajor:0 DeviceMinor:486 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2d864e2af4c097fae3a6adc8a49ec7a6bf772854923fd38a9dd162cf84666664/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-233 DeviceMajor:0 DeviceMinor:233 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:286 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9c8898951746b209c4c5d3ff9c217388d6a6b5bbbed8bb14d7ff0e46c6c609be/userdata/shm DeviceMajor:0 DeviceMinor:299 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:344 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:446 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-510 DeviceMajor:0 DeviceMinor:510 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/94135939bff49a0fa311a9f073dc3229f16aed854b2416ea53fd78eab0f2adb6/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9f075074-0903-4de4-b247-59a26f5b5f95/volumes/kubernetes.io~projected/kube-api-access-mt7bc DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7beaf096-fa6b-4233-a68b-1ab974c2e037/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:225 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-353 DeviceMajor:0 DeviceMinor:353 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/32bea234-f8f1-41fa-ac2f-bea97c729d10/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:361 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/150dd6af1f9754f9d50d8336979b00a0ab8946403e61290e66a426bc4fcf64a4/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/52797b3b-80b4-443e-aa1d-158430ee739e/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:370 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e39328c2-e756-4c99-9d13-6fbdccae91cb/volumes/kubernetes.io~projected/kube-api-access-5mwlv DeviceMajor:0 DeviceMinor:409 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:449 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-508 DeviceMajor:0 DeviceMinor:508 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7239836a-64ce-4847-91f4-9360cd652ea6/volumes/kubernetes.io~projected/kube-api-access-pzqnm DeviceMajor:0 DeviceMinor:50 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-426 DeviceMajor:0 DeviceMinor:426 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:444 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:449 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/343f8865-3015-4715-960c-a2e1418c6f7e/volumes/kubernetes.io~projected/kube-api-access-x6rf9 DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e19ce810-1d70-46db-a8ce-d824a9ffc81e/volumes/kubernetes.io~projected/kube-api-access-x8wjt DeviceMajor:0 DeviceMinor:208 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-235 DeviceMajor:0 DeviceMinor:235 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7374bdb0-a62f-4b1c-9d14-e1097c851ff0/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:255 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~projected/kube-api-access-28fm9 DeviceMajor:0 DeviceMinor:295 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-307 DeviceMajor:0 DeviceMinor:307 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/52797b3b-80b4-443e-aa1d-158430ee739e/volumes/kubernetes.io~projected/kube-api-access-sjz6h DeviceMajor:0 DeviceMinor:374 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f4a51628-0aab-4614-a4ed-11f184521cd4/volumes/kubernetes.io~projected/kube-api-access-nvb55 DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/137f70c1-c852-4c5f-823f-b8f7ed565c00/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:253 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15/volumes/kubernetes.io~projected/kube-api-access-ddw4d DeviceMajor:0 DeviceMinor:259 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:293 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:290 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e39328c2-e756-4c99-9d13-6fbdccae91cb/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:414 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:443 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7374bdb0-a62f-4b1c-9d14-e1097c851ff0/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:257 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-327 DeviceMajor:0 DeviceMinor:327 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/88dbe066e70488397c0c8a44b49766afe9dd1661739d3a6f1b29d418b982ea80/userdata/shm DeviceMajor:0 DeviceMinor:423 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:447 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~projected/kube-api-access-d92sv DeviceMajor:0 DeviceMinor:450 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:449 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:290 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/82660df8-f278-4d7c-b02a-149527e30d27/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:53 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202/volumes/kubernetes.io~projected/kube-api-access-rc46j DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:346 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c93278df-81cb-480e-9151-ba3056f43d74/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:258 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-301 DeviceMajor:0 DeviceMinor:301 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-329 DeviceMajor:0 DeviceMinor:329 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-385 DeviceMajor:0 DeviceMinor:385 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-392 DeviceMajor:0 DeviceMinor:392 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-397 DeviceMajor:0 DeviceMinor:397 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/37daf502ec6a620564fe64fcf8e755709761e5b543b611a8c11c693e07680569/userdata/shm DeviceMajor:0 DeviceMinor:203 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:341 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-349 DeviceMajor:0 DeviceMinor:349 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-379 DeviceMajor:0 DeviceMinor:379 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:445 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/915dd778-3692-4f41-83d3-cc5d4e73d615/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:47 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/80902534-76da-4e9d-8157-ce481e71f2f8/volumes/kubernetes.io~projected/kube-api-access-ghglb DeviceMajor:0 DeviceMinor:49 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f5e1af278e75b5d52d7966e63de05e330aeb88b38c16ff21c0f46ccff5bf9732/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/579c3d33f20c142fe1026324599c72f7c34a4f83fd3677b35a2ececb10451dba/userdata/shm DeviceMajor:0 DeviceMinor:268 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-305 DeviceMajor:0 DeviceMinor:305 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:449 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-464 DeviceMajor:0 DeviceMinor:464 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:297 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-466 DeviceMajor:0 DeviceMinor:466 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/af6995ea37d42bd2c4eb3ba220696c0b46e67491722cdb5ebbdbde5bad41cfb6/userdata/shm DeviceMajor:0 DeviceMinor:504 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/69524c5d81dbedb21b8902882d183dd54090d0c8bba09f1bb720201a28cc478e/userdata/shm DeviceMajor:0 DeviceMinor:505 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-514 DeviceMajor:0 DeviceMinor:514 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-516 DeviceMajor:0 DeviceMinor:516 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3f5185ad9371ea13cbae73e1eb02bfa3a76d165dcc331d716aca0b8b2ffdcc12/userdata/shm DeviceMajor:0 DeviceMinor:231 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:297 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/32bea234-f8f1-41fa-ac2f-bea97c729d10/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:360 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-434 DeviceMajor:0 DeviceMinor:434 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:448 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-481 DeviceMajor:0 DeviceMinor:481 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/efbcf64e-3ade-4343-a9c8-bf945b141a02/volumes/kubernetes.io~projected/kube-api-access-kg65k DeviceMajor:0 DeviceMinor:51 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4a065ef3d59ba12dd48cf4231807c8c00afb5d135f136d7dabfe7b53b2356aea/userdata/shm DeviceMajor:0 DeviceMinor:337 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e39328c2-e756-4c99-9d13-6fbdccae91cb/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:413 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:451 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-512 DeviceMajor:0 DeviceMinor:512 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-64 DeviceMajor:0 DeviceMinor:64 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/137f70c1-c852-4c5f-823f-b8f7ed565c00/volumes/kubernetes.io~projected/kube-api-access-c76c8 DeviceMajor:0 DeviceMinor:260 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/de37f3c4cee214cc69bffc454ecda72e0f9142c2af060ccde47e2e192b378ac7/userdata/shm DeviceMajor:0 DeviceMinor:272 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~projected/kube-api-access-9dmt9 DeviceMajor:0 DeviceMinor:347 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-476 DeviceMajor:0 DeviceMinor:476 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/379bdc95-36d7-47f2-9e17-a96e51da6495/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:501 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/343f8865-3015-4715-960c-a2e1418c6f7e/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:52 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fce1cb0777caf725040593cf1a7448767e27710547736781c8687537530dd25d/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7374bdb0-a62f-4b1c-9d14-e1097c851ff0/volumes/kubernetes.io~projected/kube-api-access-wlrrj DeviceMajor:0 DeviceMinor:261 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-243 DeviceMajor:0 DeviceMinor:243 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/137f70c1-c852-4c5f-823f-b8f7ed565c00/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:256 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-276 DeviceMajor:0 DeviceMinor:276 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:296 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:297 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/32bea234-f8f1-41fa-ac2f-bea97c729d10/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:359 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/32bea234-f8f1-41fa-ac2f-bea97c729d10/volumes/kubernetes.io~projected/kube-api-access-mrnng DeviceMajor:0 DeviceMinor:362 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/82660df8-f278-4d7c-b02a-149527e30d27/volumes/kubernetes.io~projected/kube-api-access-pjjl9 DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-462 DeviceMajor:0 DeviceMinor:462 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-111 DeviceMajor:0 DeviceMinor:111 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:292 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-402 DeviceMajor:0 DeviceMinor:402 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-421 DeviceMajor:0 DeviceMinor:421 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/88caa64b9200e78dbe39b56806babe65cda107d9b4711b1f4010c8c959b530b9/userdata/shm DeviceMajor:0 DeviceMinor:71 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-407 DeviceMajor:0 DeviceMinor:407 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bd55eee618c292174656ff666436e101d1d4a8d706c27c8dfc320f56309eed53/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-303 DeviceMajor:0 DeviceMinor:303 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:345 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:441 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:442 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/efbcf64e-3ade-4343-a9c8-bf945b141a02/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:496 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/54e8b758848496b767942024732e9c6aa18697a874cd125465400a2159011c38/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f1621b0505a7ee1a889b867df2295e11c7af927f0b50f6eaab9619290dcdfd65/userdata/shm DeviceMajor:0 DeviceMinor:262 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ca79ba46ecc6a6f60d1cc0136524237031a249b5f03a020926b52a6d7576b16d/userdata/shm DeviceMajor:0 DeviceMinor:367 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-375 DeviceMajor:0 DeviceMinor:375 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/81d3b7d850ec27e11ee9ad7f1af5b6c3f78fd747b809dddf7099656a34dbed8f/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-522 DeviceMajor:0 DeviceMinor:522 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f7b4c728-a80c-403d-88d9-af4fb0835993/volumes/kubernetes.io~projected/kube-api-access-tc4df DeviceMajor:0 DeviceMinor:48 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-278 DeviceMajor:0 DeviceMinor:278 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:343 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-351 DeviceMajor:0 DeviceMinor:351 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-527 DeviceMajor:0 DeviceMinor:527 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c93278df-81cb-480e-9151-ba3056f43d74/volumes/kubernetes.io~projected/kube-api-access-wzdc9 DeviceMajor:0 DeviceMinor:202 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e19ce810-1d70-46db-a8ce-d824a9ffc81e/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:204 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7beaf096-fa6b-4233-a68b-1ab974c2e037/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:229 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/75fd2d220aed24cc4be18d597bc21fe0ae142ab309de07a195021252a7217aa6/userdata/shm DeviceMajor:0 DeviceMinor:452 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:294 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/343f8865-3015-4715-960c-a2e1418c6f7e/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:54 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ffc140d9d862cd089d5b3cd81b402ab2707df60ab6400c5a38fef3123c5b7074/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-322 DeviceMajor:0 DeviceMinor:322 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-363 DeviceMajor:0 DeviceMinor:363 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-377 DeviceMajor:0 DeviceMinor:377 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:297 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/68869ffb949af724efeba8265deadbac63f5df90c9be986b392d98c3e12d09c1/userdata/shm DeviceMajor:0 DeviceMinor:389 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-456 DeviceMajor:0 DeviceMinor:456 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-491 DeviceMajor:0 DeviceMinor:491 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01/volumes/kubernetes.io~projected/kube-api-access-5gztg DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-415 DeviceMajor:0 DeviceMinor:415 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d96ea9b49f192b58243c43e8fafd06a4fd251b6290ffc75d0a5439abbe487bff/userdata/shm DeviceMajor:0 DeviceMinor:502 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:254 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-279 DeviceMajor:0 DeviceMinor:279 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e382c8f0-2b69-4271-8098-8328ac9b53ed/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:342 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/47be26f2-143e-4b49-824a-415e60f6140b/volumes/kubernetes.io~projected/kube-api-access-h4b8q DeviceMajor:0 DeviceMinor:500 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/90c9151a2fd867fa92e53aa4a1598e337ebe1bb24b81be305a6c9b7db4fa2aef/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/82660df8-f278-4d7c-b02a-149527e30d27/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/7beaf096-fa6b-4233-a68b-1ab974c2e037/volumes/kubernetes.io~projected/kube-api-access-xnhgf DeviceMajor:0 DeviceMinor:230 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-274 DeviceMajor:0 DeviceMinor:274 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cc61bb02f57f0ef6474fb326691082adb0a9cfd869f7df4548c847266c2dfa14/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:291 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:37daf502ec6a620 MacAddress:3a:0c:86:c7:87:28 Speed:10000 Mtu:8901} {Name:3f5185ad9371ea1 MacAddress:12:b9:9b:77:c2:2f Speed:10000 Mtu:8901} {Name:4a065ef3d59ba12 MacAddress:62:15:73:fd:0e:2a Speed:10000 Mtu:8901} {Name:54e8b758848496b MacAddress:26:a8:ea:56:d6:89 Speed:10000 Mtu:8901} {Name:68869ffb949af72 MacAddress:76:bc:4c:11:04:85 Speed:10000 Mtu:8901} {Name:69524c5d81dbedb MacAddress:92:a1:5c:de:ac:71 Speed:10000 Mtu:8901} {Name:75fd2d220aed24c MacAddress:22:e4:91:81:3c:39 Speed:10000 Mtu:8901} {Name:88dbe066e704883 MacAddress:e6:2b:b5:b5:cb:e5 Speed:10000 Mtu:8901} {Name:9c8898951746b20 MacAddress:26:98:dc:ed:c8:5f Speed:10000 Mtu:8901} {Name:af6995ea37d42bd MacAddress:a6:a7:6a:6b:ae:84 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:cf:85:a6:9d Speed:0 Mtu:9001} {Name:br-int MacAddress:f6:4f:a8:27:7d:83 Speed:0 Mtu:8901} {Name:ca79ba46ecc6a6f MacAddress:0e:52:36:0c:33:bf Speed:10000 Mtu:8901} {Name:d96ea9b49f192b5 MacAddress:d2:24:d9:34:5a:84 Speed:10000 Mtu:8901} {Name:de37f3c4cee214c MacAddress:32:f1:c3:e2:f0:d9 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:cf:85:a6:9d Speed:0 Mtu:9001} {Name:f1621b0505a7ee1 MacAddress:86:37:e7:bb:4c:07 Speed:10000 Mtu:8901} {Name:ffc140d9d862cd0 MacAddress:e2:e6:af:b4:63:c9 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:96:62:33:70:21:cc Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:0a:c9:4c:8e:e8:af Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 05:29:13.106763 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.106509 5904 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 05:29:13.106763 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.106558 5904 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 05:29:13.106907 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.106886 5904 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 05:29:13.107049 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.106908 5904 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-126.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 05:29:13.107049 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.107049 5904 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 05:29:13.107205 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.107059 5904 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 05:29:13.107205 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.107070 5904 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 05:29:13.107205 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.107089 5904 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 05:29:13.107205 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.107169 5904 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 05:29:13.107373 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.107363 5904 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 05:29:13.107481 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.107471 5904 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 05:29:13.107515 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.107508 5904 kubelet.go:493] "Attempting to sync node with API server" Nov 04 05:29:13.107542 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.107520 5904 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 05:29:13.107542 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.107537 5904 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 05:29:13.107594 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.107546 5904 kubelet.go:405] "Adding apiserver pod source" Nov 04 05:29:13.107594 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.107553 5904 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 05:29:13.108614 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.108597 5904 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 05:29:13.108614 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.108615 5904 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 05:29:13.109143 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.109124 5904 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 05:29:13.112135 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112120 5904 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 05:29:13.112135 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112134 5904 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 05:29:13.112258 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112247 5904 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Nov 04 05:29:13.112454 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112443 5904 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 05:29:13.112485 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112472 5904 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 05:29:13.112583 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112573 5904 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 05:29:13.112613 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112595 5904 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 05:29:13.112613 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112603 5904 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 05:29:13.112613 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112611 5904 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 05:29:13.112690 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112620 5904 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 05:29:13.112690 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112630 5904 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 05:29:13.112690 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112636 5904 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 05:29:13.112690 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112641 5904 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 05:29:13.112690 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112648 5904 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 05:29:13.112690 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112653 5904 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 05:29:13.112690 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112670 5904 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 05:29:13.112690 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112680 5904 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 05:29:13.112903 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112713 5904 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 05:29:13.112903 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.112722 5904 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 05:29:13.116493 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.116478 5904 server.go:1263] "Started kubelet" Nov 04 05:29:13.116580 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.116544 5904 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 05:29:13.116735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.116647 5904 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 05:29:13.117073 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.116760 5904 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 05:29:13.117425 ip-10-0-128-126 systemd[1]: Started Kubernetes Kubelet. Nov 04 05:29:13.118772 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.117445 5904 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 05:29:13.121581 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.121562 5904 server.go:317] "Adding debug handlers to kubelet server" Nov 04 05:29:13.125734 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.125715 5904 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:29:13.126788 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.126773 5904 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 05:29:13.126898 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.126793 5904 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 05:29:13.126898 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.126836 5904 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 05:29:13.126898 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.126848 5904 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 05:29:13.127039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.126899 5904 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 05:29:13.128508 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.128489 5904 factory.go:55] Registering systemd factory Nov 04 05:29:13.128508 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.128507 5904 factory.go:223] Registration of the systemd container factory successfully Nov 04 05:29:13.129454 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.129436 5904 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:29:13.129836 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.129821 5904 factory.go:153] Registering CRI-O factory Nov 04 05:29:13.129872 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.129841 5904 factory.go:223] Registration of the crio container factory successfully Nov 04 05:29:13.129911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.129902 5904 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 05:29:13.129942 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.129921 5904 factory.go:103] Registering Raw factory Nov 04 05:29:13.129942 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.129936 5904 manager.go:1199] Started watching for new ooms in manager Nov 04 05:29:13.130604 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.130318 5904 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 05:23:04 +0000 UTC" deadline="2028-08-16 01:21:31.904723382 +0000 UTC" Nov 04 05:29:13.130604 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.130346 5904 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15619h52m18.774380571s" Nov 04 05:29:13.130604 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.130366 5904 manager.go:319] Starting recovery of all containers Nov 04 05:29:13.130604 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.130483 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32bea234-f8f1-41fa-ac2f-bea97c729d10" volumeName="kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-secret-metrics-server-client-certs" seLinuxMountContext="" Nov 04 05:29:13.130604 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.130517 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="379bdc95-36d7-47f2-9e17-a96e51da6495" volumeName="kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret" seLinuxMountContext="" Nov 04 05:29:13.130604 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.130535 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47be26f2-143e-4b49-824a-415e60f6140b" volumeName="kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q" seLinuxMountContext="" Nov 04 05:29:13.130604 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.130552 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e39328c2-e756-4c99-9d13-6fbdccae91cb" volumeName="kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config" seLinuxMountContext="" Nov 04 05:29:13.130604 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.130565 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="343f8865-3015-4715-960c-a2e1418c6f7e" volumeName="kubernetes.io/empty-dir/343f8865-3015-4715-960c-a2e1418c6f7e-tmp" seLinuxMountContext="" Nov 04 05:29:13.130604 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.130580 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c93278df-81cb-480e-9151-ba3056f43d74" volumeName="kubernetes.io/secret/c93278df-81cb-480e-9151-ba3056f43d74-insights-runtime-extractor-tls" seLinuxMountContext="" Nov 04 05:29:13.130604 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.130592 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15" volumeName="kubernetes.io/projected/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-api-access-ddw4d" seLinuxMountContext="" Nov 04 05:29:13.130604 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.130605 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e382c8f0-2b69-4271-8098-8328ac9b53ed" volumeName="kubernetes.io/projected/e382c8f0-2b69-4271-8098-8328ac9b53ed-kube-api-access-9dmt9" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.130619 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7b4c728-a80c-403d-88d9-af4fb0835993" volumeName="kubernetes.io/configmap/f7b4c728-a80c-403d-88d9-af4fb0835993-iptables-alerter-script" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.133918 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82660df8-f278-4d7c-b02a-149527e30d27" volumeName="kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-ovnkube-config" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.133944 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e382c8f0-2b69-4271-8098-8328ac9b53ed" volumeName="kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-grpc-tls" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.133968 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e382c8f0-2b69-4271-8098-8328ac9b53ed" volumeName="kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.133983 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134024 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="343f8865-3015-4715-960c-a2e1418c6f7e" volumeName="kubernetes.io/projected/343f8865-3015-4715-960c-a2e1418c6f7e-kube-api-access-x6rf9" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134045 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82660df8-f278-4d7c-b02a-149527e30d27" volumeName="kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-ovnkube-script-lib" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134067 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e382c8f0-2b69-4271-8098-8328ac9b53ed" volumeName="kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134081 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e39328c2-e756-4c99-9d13-6fbdccae91cb" volumeName="kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134096 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e39328c2-e756-4c99-9d13-6fbdccae91cb" volumeName="kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134115 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-0" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134130 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32bea234-f8f1-41fa-ac2f-bea97c729d10" volumeName="kubernetes.io/empty-dir/32bea234-f8f1-41fa-ac2f-bea97c729d10-audit-log" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134149 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9f075074-0903-4de4-b247-59a26f5b5f95" volumeName="kubernetes.io/secret/9f075074-0903-4de4-b247-59a26f5b5f95-cert" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134166 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-tls" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134180 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4a51628-0aab-4614-a4ed-11f184521cd4" volumeName="kubernetes.io/projected/f4a51628-0aab-4614-a4ed-11f184521cd4-kube-api-access-nvb55" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134200 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32bea234-f8f1-41fa-ac2f-bea97c729d10" volumeName="kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-secret-metrics-server-tls" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134212 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82660df8-f278-4d7c-b02a-149527e30d27" volumeName="kubernetes.io/projected/82660df8-f278-4d7c-b02a-149527e30d27-kube-api-access-pjjl9" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134269 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15" volumeName="kubernetes.io/configmap/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-metrics-client-ca" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134283 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e39328c2-e756-4c99-9d13-6fbdccae91cb" volumeName="kubernetes.io/empty-dir/e39328c2-e756-4c99-9d13-6fbdccae91cb-tmp" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134329 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="137f70c1-c852-4c5f-823f-b8f7ed565c00" volumeName="kubernetes.io/secret/137f70c1-c852-4c5f-823f-b8f7ed565c00-openshift-state-metrics-tls" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134357 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15" volumeName="kubernetes.io/configmap/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134378 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e382c8f0-2b69-4271-8098-8328ac9b53ed" volumeName="kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134391 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e39328c2-e756-4c99-9d13-6fbdccae91cb" volumeName="kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134408 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134421 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7beaf096-fa6b-4233-a68b-1ab974c2e037" volumeName="kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134436 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e382c8f0-2b69-4271-8098-8328ac9b53ed" volumeName="kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134454 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e39328c2-e756-4c99-9d13-6fbdccae91cb" volumeName="kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134467 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dcd56353-442a-468a-88b7-6033c8686386" volumeName="kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-kube-api-access-28fm9" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134479 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e39328c2-e756-4c99-9d13-6fbdccae91cb" volumeName="kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134489 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bcebbdb-3bcd-4050-94b5-3139d6cf3d01" volumeName="kubernetes.io/projected/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-kube-api-access-5gztg" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134498 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7239836a-64ce-4847-91f4-9360cd652ea6" volumeName="kubernetes.io/projected/7239836a-64ce-4847-91f4-9360cd652ea6-kube-api-access-pzqnm" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134514 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9f075074-0903-4de4-b247-59a26f5b5f95" volumeName="kubernetes.io/projected/9f075074-0903-4de4-b247-59a26f5b5f95-kube-api-access-mt7bc" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134528 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e19ce810-1d70-46db-a8ce-d824a9ffc81e" volumeName="kubernetes.io/secret/e19ce810-1d70-46db-a8ce-d824a9ffc81e-metrics-tls" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134546 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7239836a-64ce-4847-91f4-9360cd652ea6" volumeName="kubernetes.io/configmap/7239836a-64ce-4847-91f4-9360cd652ea6-cni-binary-copy" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134560 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7beaf096-fa6b-4233-a68b-1ab974c2e037" volumeName="kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134573 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4a51628-0aab-4614-a4ed-11f184521cd4" volumeName="kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-whereabouts-flatfile-configmap" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134591 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7374bdb0-a62f-4b1c-9d14-e1097c851ff0" volumeName="kubernetes.io/configmap/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-accelerators-collector-config" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134634 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7374bdb0-a62f-4b1c-9d14-e1097c851ff0" volumeName="kubernetes.io/projected/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-kube-api-access-wlrrj" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134655 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7beaf096-fa6b-4233-a68b-1ab974c2e037" volumeName="kubernetes.io/empty-dir/7beaf096-fa6b-4233-a68b-1ab974c2e037-tmp" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134671 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dcd56353-442a-468a-88b7-6033c8686386" volumeName="kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-trusted-ca-bundle" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134685 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dcd56353-442a-468a-88b7-6033c8686386" volumeName="kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-main-db" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134705 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32bea234-f8f1-41fa-ac2f-bea97c729d10" volumeName="kubernetes.io/configmap/32bea234-f8f1-41fa-ac2f-bea97c729d10-configmap-kubelet-serving-ca-bundle" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134718 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e19ce810-1d70-46db-a8ce-d824a9ffc81e" volumeName="kubernetes.io/empty-dir/e19ce810-1d70-46db-a8ce-d824a9ffc81e-tmp-dir" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134736 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52797b3b-80b4-443e-aa1d-158430ee739e" volumeName="kubernetes.io/projected/52797b3b-80b4-443e-aa1d-158430ee739e-kube-api-access-sjz6h" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134746 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7239836a-64ce-4847-91f4-9360cd652ea6" volumeName="kubernetes.io/configmap/7239836a-64ce-4847-91f4-9360cd652ea6-multus-daemon-config" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.133391 5904 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134762 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7374bdb0-a62f-4b1c-9d14-e1097c851ff0" volumeName="kubernetes.io/secret/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-tls" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134812 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7beaf096-fa6b-4233-a68b-1ab974c2e037" volumeName="kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134848 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="137f70c1-c852-4c5f-823f-b8f7ed565c00" volumeName="kubernetes.io/configmap/137f70c1-c852-4c5f-823f-b8f7ed565c00-metrics-client-ca" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134890 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="915dd778-3692-4f41-83d3-cc5d4e73d615" volumeName="kubernetes.io/secret/915dd778-3692-4f41-83d3-cc5d4e73d615-agent-certs" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134930 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="80902534-76da-4e9d-8157-ce481e71f2f8" volumeName="kubernetes.io/projected/80902534-76da-4e9d-8157-ce481e71f2f8-kube-api-access-ghglb" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134947 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15" volumeName="kubernetes.io/empty-dir/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-volume-directive-shadow" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134966 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-serving-certs-ca-bundle" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.134980 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-kubelet-serving-ca-bundle" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135014 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-kube-api-access-d92sv" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135028 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-thanos-prometheus-http-client-file" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135041 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-grpc-tls" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135059 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-web-config" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135091 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="137f70c1-c852-4c5f-823f-b8f7ed565c00" volumeName="kubernetes.io/secret/137f70c1-c852-4c5f-823f-b8f7ed565c00-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135108 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-db" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135122 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32bea234-f8f1-41fa-ac2f-bea97c729d10" volumeName="kubernetes.io/configmap/32bea234-f8f1-41fa-ac2f-bea97c729d10-metrics-server-audit-profiles" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135136 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dcd56353-442a-468a-88b7-6033c8686386" volumeName="kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135170 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dcd56353-442a-468a-88b7-6033c8686386" volumeName="kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-main-tls" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135184 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-kube-rbac-proxy" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135200 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c93278df-81cb-480e-9151-ba3056f43d74" volumeName="kubernetes.io/projected/c93278df-81cb-480e-9151-ba3056f43d74-kube-api-access-wzdc9" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135213 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e39328c2-e756-4c99-9d13-6fbdccae91cb" volumeName="kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135269 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4a51628-0aab-4614-a4ed-11f184521cd4" volumeName="kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-cni-binary-copy" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135294 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dcd56353-442a-468a-88b7-6033c8686386" volumeName="kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-config-out" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135313 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7b4c728-a80c-403d-88d9-af4fb0835993" volumeName="kubernetes.io/projected/f7b4c728-a80c-403d-88d9-af4fb0835993-kube-api-access-tc4df" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135372 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7374bdb0-a62f-4b1c-9d14-e1097c851ff0" volumeName="kubernetes.io/secret/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-kube-rbac-proxy-config" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135390 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7374bdb0-a62f-4b1c-9d14-e1097c851ff0" volumeName="kubernetes.io/configmap/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-metrics-client-ca" seLinuxMountContext="" Nov 04 05:29:13.135427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135483 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7374bdb0-a62f-4b1c-9d14-e1097c851ff0" volumeName="kubernetes.io/empty-dir/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-textfile" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135495 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c93278df-81cb-480e-9151-ba3056f43d74" volumeName="kubernetes.io/configmap/c93278df-81cb-480e-9151-ba3056f43d74-kube-rbac-proxy-cm" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135510 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-1" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135524 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="343f8865-3015-4715-960c-a2e1418c6f7e" volumeName="kubernetes.io/empty-dir/343f8865-3015-4715-960c-a2e1418c6f7e-etc-tuned" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135540 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7beaf096-fa6b-4233-a68b-1ab974c2e037" volumeName="kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135550 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82660df8-f278-4d7c-b02a-149527e30d27" volumeName="kubernetes.io/secret/82660df8-f278-4d7c-b02a-149527e30d27-ovn-node-metrics-cert" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135588 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dcd56353-442a-468a-88b7-6033c8686386" volumeName="kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135622 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7beaf096-fa6b-4233-a68b-1ab974c2e037" volumeName="kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135631 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c93278df-81cb-480e-9151-ba3056f43d74" volumeName="kubernetes.io/empty-dir/c93278df-81cb-480e-9151-ba3056f43d74-data-volume" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135642 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-trusted-ca-bundle" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135670 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32bea234-f8f1-41fa-ac2f-bea97c729d10" volumeName="kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-client-ca-bundle" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135691 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dcd56353-442a-468a-88b7-6033c8686386" volumeName="kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-tls-assets" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135746 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dcd56353-442a-468a-88b7-6033c8686386" volumeName="kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-web-config" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135758 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e382c8f0-2b69-4271-8098-8328ac9b53ed" volumeName="kubernetes.io/configmap/e382c8f0-2b69-4271-8098-8328ac9b53ed-metrics-client-ca" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135768 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e382c8f0-2b69-4271-8098-8328ac9b53ed" volumeName="kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-tls" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135776 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="efbcf64e-3ade-4343-a9c8-bf945b141a02" volumeName="kubernetes.io/projected/efbcf64e-3ade-4343-a9c8-bf945b141a02-kube-api-access-kg65k" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135792 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dbbe9ebe-9bf8-43fd-88ff-8b5b23126202" volumeName="kubernetes.io/projected/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-kube-api-access-rc46j" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135806 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-metrics-client-certs" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135828 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="137f70c1-c852-4c5f-823f-b8f7ed565c00" volumeName="kubernetes.io/projected/137f70c1-c852-4c5f-823f-b8f7ed565c00-kube-api-access-c76c8" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135840 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dcd56353-442a-468a-88b7-6033c8686386" volumeName="kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135857 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15" volumeName="kubernetes.io/secret/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135867 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-2" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135895 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-tls-assets" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135907 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7beaf096-fa6b-4233-a68b-1ab974c2e037" volumeName="kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135935 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82660df8-f278-4d7c-b02a-149527e30d27" volumeName="kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-env-overrides" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135962 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dcd56353-442a-468a-88b7-6033c8686386" volumeName="kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-cluster-tls-config" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.135979 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15" volumeName="kubernetes.io/secret/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-tls" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136023 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-config" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136053 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="efbcf64e-3ade-4343-a9c8-bf945b141a02" volumeName="kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136069 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4a51628-0aab-4614-a4ed-11f184521cd4" volumeName="kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-cni-sysctl-allowlist" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136103 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bcebbdb-3bcd-4050-94b5-3139d6cf3d01" volumeName="kubernetes.io/configmap/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-serviceca" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136143 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dcd56353-442a-468a-88b7-6033c8686386" volumeName="kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-metrics-client-ca" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136158 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e19ce810-1d70-46db-a8ce-d824a9ffc81e" volumeName="kubernetes.io/projected/e19ce810-1d70-46db-a8ce-d824a9ffc81e-kube-api-access-x8wjt" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136178 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32bea234-f8f1-41fa-ac2f-bea97c729d10" volumeName="kubernetes.io/projected/32bea234-f8f1-41fa-ac2f-bea97c729d10-kube-api-access-mrnng" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136258 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="80902534-76da-4e9d-8157-ce481e71f2f8" volumeName="kubernetes.io/empty-dir/80902534-76da-4e9d-8157-ce481e71f2f8-tmp-dir" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136281 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dcd56353-442a-468a-88b7-6033c8686386" volumeName="kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-config-volume" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136323 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52797b3b-80b4-443e-aa1d-158430ee739e" volumeName="kubernetes.io/secret/52797b3b-80b4-443e-aa1d-158430ee739e-monitoring-plugin-cert" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136342 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e382c8f0-2b69-4271-8098-8328ac9b53ed" volumeName="kubernetes.io/empty-dir/e382c8f0-2b69-4271-8098-8328ac9b53ed-tmp" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136351 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-config-out" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136374 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7beaf096-fa6b-4233-a68b-1ab974c2e037" volumeName="kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136390 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="915dd778-3692-4f41-83d3-cc5d4e73d615" volumeName="kubernetes.io/configmap/915dd778-3692-4f41-83d3-cc5d4e73d615-konnectivity-ca" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136407 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e19ce810-1d70-46db-a8ce-d824a9ffc81e" volumeName="kubernetes.io/configmap/e19ce810-1d70-46db-a8ce-d824a9ffc81e-config-volume" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136464 5904 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e402f928-dd6c-4f77-acbb-94b57e6b5188" volumeName="kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-metrics-client-ca" seLinuxMountContext="" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136489 5904 reconstruct.go:97] "Volume reconstruction finished" Nov 04 05:29:13.138881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.136496 5904 reconciler.go:29] "Reconciler: start to sync state" Nov 04 05:29:13.140832 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.139741 5904 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Nov 04 05:29:13.157735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.157712 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-slrx2_82660df8-f278-4d7c-b02a-149527e30d27/ovn-acl-logging/0.log" Nov 04 05:29:13.159508 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.159459 5904 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 05:29:13.161364 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.161021 5904 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 05:29:13.161364 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.161043 5904 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 05:29:13.161364 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.161069 5904 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 05:29:13.161364 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:13.161109 5904 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Nov 04 05:29:13.164455 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.164434 5904 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:29:13.164934 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.164910 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-slrx2_82660df8-f278-4d7c-b02a-149527e30d27/ovn-acl-logging/0.log" Nov 04 05:29:13.182397 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.182364 5904 generic.go:356] "Generic (PLEG): container finished" podID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerID="3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb" exitCode=0 Nov 04 05:29:13.198683 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.198647 5904 generic.go:356] "Generic (PLEG): container finished" podID="f4a51628-0aab-4614-a4ed-11f184521cd4" containerID="1a45106a3e204439c36c8d181bebb9116c9b2b5b0c33c91759de2e08c4adae11" exitCode=0 Nov 04 05:29:13.198683 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.198680 5904 generic.go:356] "Generic (PLEG): container finished" podID="f4a51628-0aab-4614-a4ed-11f184521cd4" containerID="c9cc9d148cc1991401cc7deb85c90ff87bbb4c12c2747bcaf3f7b3792781c11c" exitCode=0 Nov 04 05:29:13.198683 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.198689 5904 generic.go:356] "Generic (PLEG): container finished" podID="f4a51628-0aab-4614-a4ed-11f184521cd4" containerID="00d8a6c2e92669146e386a88bca65888b34ea3b007480f849f2d44509b5224c5" exitCode=0 Nov 04 05:29:13.198892 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.198699 5904 generic.go:356] "Generic (PLEG): container finished" podID="f4a51628-0aab-4614-a4ed-11f184521cd4" containerID="79313f40b83df9ed661abc441aade08d8646e7c2e037298d783a951e6cde2ffa" exitCode=0 Nov 04 05:29:13.198892 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.198708 5904 generic.go:356] "Generic (PLEG): container finished" podID="f4a51628-0aab-4614-a4ed-11f184521cd4" containerID="a3698660193295b886168c0a699a62446521b18a6d4b8f3f8c11a2e4fcdcbecf" exitCode=0 Nov 04 05:29:13.198892 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.198715 5904 generic.go:356] "Generic (PLEG): container finished" podID="f4a51628-0aab-4614-a4ed-11f184521cd4" containerID="fbf7189995ced8b432ac08e3fe4c38b967eda51c6ec4a3297345a7f826200502" exitCode=0 Nov 04 05:29:13.212664 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.212627 5904 generic.go:356] "Generic (PLEG): container finished" podID="dcd56353-442a-468a-88b7-6033c8686386" containerID="b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86" exitCode=0 Nov 04 05:29:13.214899 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.214870 5904 generic.go:356] "Generic (PLEG): container finished" podID="7374bdb0-a62f-4b1c-9d14-e1097c851ff0" containerID="161adae46962bdf9e89da45baba3fca1a98ecb0afc296dc3bb98407df913995b" exitCode=0 Nov 04 05:29:13.236094 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.236037 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-slrx2_82660df8-f278-4d7c-b02a-149527e30d27/ovn-acl-logging/0.log" Nov 04 05:29:13.236400 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.236368 5904 generic.go:356] "Generic (PLEG): container finished" podID="82660df8-f278-4d7c-b02a-149527e30d27" containerID="0caaa4108d97a74bdc828247da5f268f726e7bb7e92925f3e1c1202384f01e02" exitCode=1 Nov 04 05:29:13.239918 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.239893 5904 generic.go:356] "Generic (PLEG): container finished" podID="58571c34b4dbebe904f2b5a148e16c40" containerID="89f7e054c5a795fefa73216b0248ffbf88d728ca35f5dae8f5f9d551b556b29b" exitCode=0 Nov 04 05:29:13.259031 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.259011 5904 manager.go:324] Recovery completed Nov 04 05:29:13.261161 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:13.261142 5904 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Nov 04 05:29:13.284433 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.284410 5904 cpu_manager.go:226] "Starting" policy="none" Nov 04 05:29:13.284433 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.284425 5904 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 05:29:13.284584 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.284444 5904 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 05:29:13.284653 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.284584 5904 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Nov 04 05:29:13.284653 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.284595 5904 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Nov 04 05:29:13.284653 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.284612 5904 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Nov 04 05:29:13.284653 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.284618 5904 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Nov 04 05:29:13.284653 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.284623 5904 policy_none.go:50] "Start" Nov 04 05:29:13.284653 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.284633 5904 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 05:29:13.284653 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.284643 5904 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 05:29:13.284886 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.284769 5904 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Nov 04 05:29:13.284886 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.284775 5904 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Nov 04 05:29:13.284886 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.284781 5904 policy_none.go:44] "Start" Nov 04 05:29:13.289589 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.289560 5904 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 05:29:13.292868 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.292855 5904 manager.go:341] "Starting Device Plugin manager" Nov 04 05:29:13.292938 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:13.292885 5904 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 05:29:13.292938 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.292899 5904 server.go:87] "Starting device plugin registration server" Nov 04 05:29:13.293630 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.293613 5904 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 05:29:13.293702 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.293630 5904 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 05:29:13.293702 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.293694 5904 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 05:29:13.293824 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.293779 5904 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 05:29:13.293875 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.293830 5904 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 05:29:13.293875 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.293837 5904 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 05:29:13.293969 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.293890 5904 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T05:29:13.293820641Z","UUID":"7aa8056b-4438-40bc-b86a-05b765bfd1d9","Handler":null,"Name":"","Endpoint":""} Nov 04 05:29:13.295733 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:13.295705 5904 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 05:29:13.295829 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.295805 5904 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 05:29:13.295886 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.295831 5904 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 05:29:13.393886 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.393856 5904 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:29:13.396614 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.396590 5904 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-126.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:29:13.396704 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.396623 5904 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-126.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:29:13.396704 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.396633 5904 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-126.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:29:13.396704 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.396682 5904 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:13.409669 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.409645 5904 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:13.409741 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.409711 5904 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:13.461685 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.461638 5904 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal"] Nov 04 05:29:13.462340 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.462326 5904 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:13.462414 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.462355 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" event={"ID":"fb1cdcff7ca15ea5212ef7ed52c2b8e3","Type":"ContainerStarted","Data":"9802c542b62766d90de88595528428a4ebb10e446916a80e13f5815033bccc9b"} Nov 04 05:29:13.462449 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.462414 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" event={"ID":"fb1cdcff7ca15ea5212ef7ed52c2b8e3","Type":"ContainerStarted","Data":"88caa64b9200e78dbe39b56806babe65cda107d9b4711b1f4010c8c959b530b9"} Nov 04 05:29:13.462563 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.462546 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" event={"ID":"58571c34b4dbebe904f2b5a148e16c40","Type":"ContainerStarted","Data":"af400f861c41f3a1bfc6a2f1fd869d39df5a49f001297922376d2384881275e5"} Nov 04 05:29:13.462602 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.462577 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" event={"ID":"58571c34b4dbebe904f2b5a148e16c40","Type":"ContainerDied","Data":"89f7e054c5a795fefa73216b0248ffbf88d728ca35f5dae8f5f9d551b556b29b"} Nov 04 05:29:13.462602 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.462593 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" event={"ID":"58571c34b4dbebe904f2b5a148e16c40","Type":"ContainerStarted","Data":"cc61bb02f57f0ef6474fb326691082adb0a9cfd869f7df4548c847266c2dfa14"} Nov 04 05:29:13.462668 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.462555 5904 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:13.479095 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.479073 5904 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:29:13.479168 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:13.479110 5904 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:13.479688 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.479665 5904 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:29:13.479725 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:13.479715 5904 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:13.542052 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.541983 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58571c34b4dbebe904f2b5a148e16c40-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"58571c34b4dbebe904f2b5a148e16c40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:13.542052 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.542043 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fb1cdcff7ca15ea5212ef7ed52c2b8e3-config\") pod \"kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"fb1cdcff7ca15ea5212ef7ed52c2b8e3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:13.542190 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.542110 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58571c34b4dbebe904f2b5a148e16c40-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"58571c34b4dbebe904f2b5a148e16c40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:13.642610 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.642577 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58571c34b4dbebe904f2b5a148e16c40-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"58571c34b4dbebe904f2b5a148e16c40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:13.642610 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.642606 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58571c34b4dbebe904f2b5a148e16c40-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"58571c34b4dbebe904f2b5a148e16c40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:13.642767 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.642625 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58571c34b4dbebe904f2b5a148e16c40-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"58571c34b4dbebe904f2b5a148e16c40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:13.642767 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.642662 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58571c34b4dbebe904f2b5a148e16c40-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"58571c34b4dbebe904f2b5a148e16c40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:13.642767 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.642664 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fb1cdcff7ca15ea5212ef7ed52c2b8e3-config\") pod \"kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"fb1cdcff7ca15ea5212ef7ed52c2b8e3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:13.642767 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:13.642698 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fb1cdcff7ca15ea5212ef7ed52c2b8e3-config\") pod \"kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal\" (UID: \"fb1cdcff7ca15ea5212ef7ed52c2b8e3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:14.108483 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.108450 5904 apiserver.go:52] "Watching apiserver" Nov 04 05:29:14.197639 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.197610 5904 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 05:29:14.198360 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.198334 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-fvxf5","openshift-monitoring/alertmanager-main-0","openshift-monitoring/node-exporter-dzs66","openshift-image-registry/node-ca-rkg7g","kube-system/konnectivity-agent-m66r7","openshift-insights/insights-runtime-extractor-skkvk","kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx","openshift-monitoring/monitoring-plugin-77f4f76594-4s58k","openshift-monitoring/thanos-querier-99867478d-zstlh","openshift-network-diagnostics/network-check-target-mq5nl","openshift-network-operator/iptables-alerter-m77gq","openshift-cluster-node-tuning-operator/tuned-gbms4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal","openshift-monitoring/metrics-server-765454fc56-b5f6j","openshift-ovn-kubernetes/ovnkube-node-slrx2","kube-system/global-pull-secret-syncer-69rl9","openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647","openshift-multus/multus-additional-cni-plugins-z9djg","openshift-console/console-6db77f4495-958wf","openshift-multus/network-metrics-daemon-brh9c","openshift-ingress-canary/ingress-canary-2k9r5","openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd","openshift-monitoring/prometheus-k8s-0","openshift-multus/multus-hk726","openshift-dns/dns-default-9zfz8","openshift-console/console-5f5f49d4fc-6cgqw"] Nov 04 05:29:14.202842 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.202817 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 05:29:14.202957 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.202928 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 05:29:14.203283 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.203263 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 05:29:14.204832 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.204809 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-67qcf\"" Nov 04 05:29:14.204958 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.204844 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pqp2n\"" Nov 04 05:29:14.204958 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.204853 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-4fjpb\"" Nov 04 05:29:14.204958 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.204873 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 05:29:14.204958 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.204911 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 05:29:14.204958 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.204918 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xrb2c\"" Nov 04 05:29:14.204958 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.204927 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 05:29:14.205207 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.204972 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 05:29:14.205207 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.204977 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 05:29:14.205207 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.205024 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7x6ks\"" Nov 04 05:29:14.205207 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.205065 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 05:29:14.205415 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.205394 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 05:29:14.205512 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.205437 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 05:29:14.205512 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.205462 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 05:29:14.205582 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.205518 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:29:14.205736 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.205720 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rpj2j\"" Nov 04 05:29:14.206171 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.206151 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 05:29:14.206171 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.206153 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-m8rqh\"" Nov 04 05:29:14.206341 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.206215 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 05:29:14.206410 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.206351 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zs9xj\"" Nov 04 05:29:14.206410 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.206355 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 05:29:14.206410 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.206383 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 05:29:14.206556 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.206448 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-kkpcg\"" Nov 04 05:29:14.206843 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.206820 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 05:29:14.206939 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.206863 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 05:29:14.206939 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.206893 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 05:29:14.208953 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.208931 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 05:29:14.229594 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.229572 5904 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 05:29:14.243306 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.243284 5904 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:14.243413 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.243388 5904 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:14.245574 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.245554 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7239836a-64ce-4847-91f4-9360cd652ea6-cni-binary-copy\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.245667 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.245588 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.245667 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.245617 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gztg\" (UniqueName: \"kubernetes.io/projected/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-kube-api-access-5gztg\") pod \"node-ca-rkg7g\" (UID: \"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01\") " pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:29:14.245667 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.245644 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4b8q\" (UniqueName: \"kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q\") pod \"network-check-target-mq5nl\" (UID: \"47be26f2-143e-4b49-824a-415e60f6140b\") " pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:29:14.245810 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.245672 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-ovn\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.245810 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.245698 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-cni-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.245810 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.245725 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e19ce810-1d70-46db-a8ce-d824a9ffc81e-config-volume\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:29:14.245810 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.245753 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.245810 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.245753 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.245810 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.245801 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7239836a-64ce-4847-91f4-9360cd652ea6-cni-binary-copy\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.246009 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.245801 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvb55\" (UniqueName: \"kubernetes.io/projected/f4a51628-0aab-4614-a4ed-11f184521cd4-kube-api-access-nvb55\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.246009 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.245903 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c93278df-81cb-480e-9151-ba3056f43d74-crio-socket\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:29:14.246009 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.245944 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-registration-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.246009 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.245963 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-system-cni-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.246009 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.245982 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:29:14.246191 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246045 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:14.246191 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246107 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/137f70c1-c852-4c5f-823f-b8f7ed565c00-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:29:14.246191 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.246137 5904 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:14.246353 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.246208 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:14.746193122 +0000 UTC m=+1.703778078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:14.246353 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246277 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:14.246353 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246311 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-sys-fs\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.246353 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246340 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt7bc\" (UniqueName: \"kubernetes.io/projected/9f075074-0903-4de4-b247-59a26f5b5f95-kube-api-access-mt7bc\") pod \"ingress-canary-2k9r5\" (UID: \"9f075074-0903-4de4-b247-59a26f5b5f95\") " pod="openshift-ingress-canary/ingress-canary-2k9r5" Nov 04 05:29:14.246564 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.246357 5904 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:14.246564 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246380 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.246564 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246401 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.246564 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.246418 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:14.746400731 +0000 UTC m=+1.703985681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:14.246564 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246416 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/379bdc95-36d7-47f2-9e17-a96e51da6495-original-pull-secret\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:29:14.246564 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246447 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-hostroot\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.246564 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246482 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8wjt\" (UniqueName: \"kubernetes.io/projected/e19ce810-1d70-46db-a8ce-d824a9ffc81e-kube-api-access-x8wjt\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:29:14.246564 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246511 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.246564 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246544 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mwlv\" (UniqueName: \"kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:14.246564 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246563 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-tls\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.246930 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246585 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysconfig\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.246930 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246608 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-client-ca-bundle\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:29:14.246930 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246626 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrnng\" (UniqueName: \"kubernetes.io/projected/32bea234-f8f1-41fa-ac2f-bea97c729d10-kube-api-access-mrnng\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:29:14.246930 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246641 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/379bdc95-36d7-47f2-9e17-a96e51da6495-dbus\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:29:14.246930 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246695 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.246930 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246796 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.246930 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246862 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dmt9\" (UniqueName: \"kubernetes.io/projected/e382c8f0-2b69-4271-8098-8328ac9b53ed-kube-api-access-9dmt9\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.247115 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.246968 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/82660df8-f278-4d7c-b02a-149527e30d27-ovn-node-metrics-cert\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.247115 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247048 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-metrics-client-ca\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.247115 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247067 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.247115 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247093 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-cni-bin\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.247115 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247110 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80902534-76da-4e9d-8157-ce481e71f2f8-tmp-dir\") pod \"node-resolver-fvxf5\" (UID: \"80902534-76da-4e9d-8157-ce481e71f2f8\") " pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:29:14.247326 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247135 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/379bdc95-36d7-47f2-9e17-a96e51da6495-kubelet-config\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:29:14.247326 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247161 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.247326 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247186 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7b4c728-a80c-403d-88d9-af4fb0835993-iptables-alerter-script\") pod \"iptables-alerter-m77gq\" (UID: \"f7b4c728-a80c-403d-88d9-af4fb0835993\") " pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:29:14.247326 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247209 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52797b3b-80b4-443e-aa1d-158430ee739e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-4s58k\" (UID: \"52797b3b-80b4-443e-aa1d-158430ee739e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" Nov 04 05:29:14.247326 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247214 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80902534-76da-4e9d-8157-ce481e71f2f8-tmp-dir\") pod \"node-resolver-fvxf5\" (UID: \"80902534-76da-4e9d-8157-ce481e71f2f8\") " pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:29:14.247326 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247247 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-run-netns\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.247326 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247273 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-multus-certs\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.247326 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247291 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.247645 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247335 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.247645 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247366 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c93278df-81cb-480e-9151-ba3056f43d74-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:29:14.247645 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247392 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:14.247645 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247409 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-systemd-units\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.247645 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247425 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysctl-d\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.247645 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247440 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-netns\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.247645 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247457 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-cni-binary-copy\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.247645 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247488 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc4df\" (UniqueName: \"kubernetes.io/projected/f7b4c728-a80c-403d-88d9-af4fb0835993-kube-api-access-tc4df\") pod \"iptables-alerter-m77gq\" (UID: \"f7b4c728-a80c-403d-88d9-af4fb0835993\") " pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:29:14.247645 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.247575 5904 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:14.247645 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247585 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/137f70c1-c852-4c5f-823f-b8f7ed565c00-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:29:14.247645 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247614 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e382c8f0-2b69-4271-8098-8328ac9b53ed-metrics-client-ca\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.247645 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.247634 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:14.747622739 +0000 UTC m=+1.705207669 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247658 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/915dd778-3692-4f41-83d3-cc5d4e73d615-konnectivity-ca\") pod \"konnectivity-agent-m66r7\" (UID: \"915dd778-3692-4f41-83d3-cc5d4e73d615\") " pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247678 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/343f8865-3015-4715-960c-a2e1418c6f7e-etc-tuned\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247713 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-cni-binary-copy\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247694 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-cni-multus\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247736 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/343f8865-3015-4715-960c-a2e1418c6f7e-etc-tuned\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247747 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247789 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247803 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/915dd778-3692-4f41-83d3-cc5d4e73d615-konnectivity-ca\") pod \"konnectivity-agent-m66r7\" (UID: \"915dd778-3692-4f41-83d3-cc5d4e73d615\") " pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247819 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247847 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjjl9\" (UniqueName: \"kubernetes.io/projected/82660df8-f278-4d7c-b02a-149527e30d27-kube-api-access-pjjl9\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.247857 5904 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247876 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-kubelet\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.247898 5904 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.247905 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:14.747892569 +0000 UTC m=+1.705477512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247920 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247940 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.247954 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:14.747939479 +0000 UTC m=+1.705524421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.247983 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248011 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248043 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/915dd778-3692-4f41-83d3-cc5d4e73d615-agent-certs\") pod \"konnectivity-agent-m66r7\" (UID: \"915dd778-3692-4f41-83d3-cc5d4e73d615\") " pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.248068 5904 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248070 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-wtmp\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.248094 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:14.748084677 +0000 UTC m=+1.705669636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-config" not registered Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248105 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-config-out\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248124 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248150 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28fm9\" (UniqueName: \"kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-kube-api-access-28fm9\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248174 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-conf-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.248194 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248192 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-config-out\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248240 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248257 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/915dd778-3692-4f41-83d3-cc5d4e73d615-agent-certs\") pod \"konnectivity-agent-m66r7\" (UID: \"915dd778-3692-4f41-83d3-cc5d4e73d615\") " pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248272 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-serviceca\") pod \"node-ca-rkg7g\" (UID: \"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01\") " pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248301 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248329 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248371 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c93278df-81cb-480e-9151-ba3056f43d74-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248440 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248472 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjz6h\" (UniqueName: \"kubernetes.io/projected/52797b3b-80b4-443e-aa1d-158430ee739e-kube-api-access-sjz6h\") pod \"monitoring-plugin-77f4f76594-4s58k\" (UID: \"52797b3b-80b4-443e-aa1d-158430ee739e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248500 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-node-log\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248529 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248553 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248565 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-root\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248617 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248654 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysctl-conf\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248680 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-lib-modules\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248704 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-var-lib-kubelet\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248729 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-socket-dir-parent\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248757 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzqnm\" (UniqueName: \"kubernetes.io/projected/7239836a-64ce-4847-91f4-9360cd652ea6-kube-api-access-pzqnm\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248796 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e19ce810-1d70-46db-a8ce-d824a9ffc81e-tmp-dir\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248819 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248850 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-serviceca\") pod \"node-ca-rkg7g\" (UID: \"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01\") " pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248845 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248906 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e19ce810-1d70-46db-a8ce-d824a9ffc81e-tmp-dir\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248911 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-cnibin\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248952 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248974 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/137f70c1-c852-4c5f-823f-b8f7ed565c00-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.248987 5904 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.249050 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:14.74903607 +0000 UTC m=+1.706621017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.248990 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-sys\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249095 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-grpc-tls\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249118 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddw4d\" (UniqueName: \"kubernetes.io/projected/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-api-access-ddw4d\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249134 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e39328c2-e756-4c99-9d13-6fbdccae91cb-tmp\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:14.249154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249156 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-etc-selinux\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249197 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-log-socket\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249220 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249263 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-kubernetes\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249301 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e19ce810-1d70-46db-a8ce-d824a9ffc81e-metrics-tls\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249370 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249378 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e39328c2-e756-4c99-9d13-6fbdccae91cb-tmp\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249387 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249474 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249495 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-ovnkube-script-lib\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249514 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-tls\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249531 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249575 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249605 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7beaf096-fa6b-4233-a68b-1ab974c2e037-tmp\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249634 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-var-lib-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.249653 5904 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249665 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249702 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-run\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.249708 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:14.749695218 +0000 UTC m=+1.707280163 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249657 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7beaf096-fa6b-4233-a68b-1ab974c2e037-tmp\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249768 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-cnibin\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249801 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-config-out\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249877 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.249932 5904 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249951 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-config-out\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.249954 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-systemd\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.249973 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:14.749959891 +0000 UTC m=+1.707544843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"service-ca" not registered Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250007 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-ovnkube-config\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250035 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-modprobe-d\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250137 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-host\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250155 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghglb\" (UniqueName: \"kubernetes.io/projected/80902534-76da-4e9d-8157-ce481e71f2f8-kube-api-access-ghglb\") pod \"node-resolver-fvxf5\" (UID: \"80902534-76da-4e9d-8157-ce481e71f2f8\") " pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250173 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250188 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e382c8f0-2b69-4271-8098-8328ac9b53ed-tmp\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250190 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-ovnkube-config\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250222 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-env-overrides\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250281 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7b4c728-a80c-403d-88d9-af4fb0835993-host-slash\") pod \"iptables-alerter-m77gq\" (UID: \"f7b4c728-a80c-403d-88d9-af4fb0835993\") " pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250298 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-run-ovn-kubernetes\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250315 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e382c8f0-2b69-4271-8098-8328ac9b53ed-tmp\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250348 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-config-volume\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.250435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250373 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-secret-metrics-server-tls\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250474 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32bea234-f8f1-41fa-ac2f-bea97c729d10-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250493 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/32bea234-f8f1-41fa-ac2f-bea97c729d10-audit-log\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250510 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250525 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-cni-netd\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250565 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-etc-kubernetes\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250586 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/32bea234-f8f1-41fa-ac2f-bea97c729d10-metrics-server-audit-profiles\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250602 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-config\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250617 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-system-cni-dir\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250751 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-slash\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250801 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/32bea234-f8f1-41fa-ac2f-bea97c729d10-audit-log\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250842 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-textfile\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250872 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-accelerators-collector-config\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250899 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-textfile\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250902 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250944 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg65k\" (UniqueName: \"kubernetes.io/projected/efbcf64e-3ade-4343-a9c8-bf945b141a02-kube-api-access-kg65k\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250974 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.250994 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251013 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-kubelet\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251038 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-etc-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.251090 5904 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251100 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-web-config\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251137 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-systemd\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.251277 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:14.75126305 +0000 UTC m=+1.708847996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-config" not registered Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251301 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f075074-0903-4de4-b247-59a26f5b5f95-cert\") pod \"ingress-canary-2k9r5\" (UID: \"9f075074-0903-4de4-b247-59a26f5b5f95\") " pod="openshift-ingress-canary/ingress-canary-2k9r5" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251336 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlrrj\" (UniqueName: \"kubernetes.io/projected/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-kube-api-access-wlrrj\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251374 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6rf9\" (UniqueName: \"kubernetes.io/projected/343f8865-3015-4715-960c-a2e1418c6f7e-kube-api-access-x6rf9\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251398 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-k8s-cni-cncf-io\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251430 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251457 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251476 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d92sv\" (UniqueName: \"kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-kube-api-access-d92sv\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251501 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-host\") pod \"node-ca-rkg7g\" (UID: \"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01\") " pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251547 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251576 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-socket-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.251612 5904 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251656 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-device-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.251690 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:14.751679137 +0000 UTC m=+1.709264092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"service-ca" not registered Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251708 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc46j\" (UniqueName: \"kubernetes.io/projected/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-kube-api-access-rc46j\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251726 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/343f8865-3015-4715-960c-a2e1418c6f7e-tmp\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251743 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7239836a-64ce-4847-91f4-9360cd652ea6-multus-daemon-config\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251759 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-web-config\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251774 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-sys\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251792 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251819 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c93278df-81cb-480e-9151-ba3056f43d74-data-volume\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251843 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c76c8\" (UniqueName: \"kubernetes.io/projected/137f70c1-c852-4c5f-823f-b8f7ed565c00-kube-api-access-c76c8\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251853 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/343f8865-3015-4715-960c-a2e1418c6f7e-tmp\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251861 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/80902534-76da-4e9d-8157-ce481e71f2f8-hosts-file\") pod \"node-resolver-fvxf5\" (UID: \"80902534-76da-4e9d-8157-ce481e71f2f8\") " pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:29:14.251867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251882 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251931 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c93278df-81cb-480e-9151-ba3056f43d74-data-volume\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251975 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251991 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7239836a-64ce-4847-91f4-9360cd652ea6-multus-daemon-config\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.251996 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-cni-bin\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.252017 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.252088 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.252111 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.252267 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnhgf\" (UniqueName: \"kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.252295 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f4a51628-0aab-4614-a4ed-11f184521cd4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.252302 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.252330 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.252348 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-secret-metrics-server-client-certs\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.252351 5904 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.252370 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.252389 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:14.752378356 +0000 UTC m=+1.709963287 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.252395 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efbcf64e-3ade-4343-a9c8-bf945b141a02-metrics-certs\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:29:14.253372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.252420 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzdc9\" (UniqueName: \"kubernetes.io/projected/c93278df-81cb-480e-9151-ba3056f43d74-kube-api-access-wzdc9\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:29:14.253891 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.253872 5904 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:29:14.253922 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.253906 5904 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:29:14.253950 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.253929 5904 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:14.253980 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.253931 5904 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal" Nov 04 05:29:14.255921 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.255905 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvb55\" (UniqueName: \"kubernetes.io/projected/f4a51628-0aab-4614-a4ed-11f184521cd4-kube-api-access-nvb55\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.256054 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.256036 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gztg\" (UniqueName: \"kubernetes.io/projected/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-kube-api-access-5gztg\") pod \"node-ca-rkg7g\" (UID: \"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01\") " pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:29:14.256201 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.256187 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4b8q\" (UniqueName: \"kubernetes.io/projected/47be26f2-143e-4b49-824a-415e60f6140b-kube-api-access-h4b8q\") pod \"network-check-target-mq5nl\" (UID: \"47be26f2-143e-4b49-824a-415e60f6140b\") " pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:29:14.257010 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.256995 5904 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 04 05:29:14.257048 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.257013 5904 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 04 05:29:14.257048 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.257025 5904 projected.go:196] Error preparing data for projected volume kube-api-access-5mwlv for pod openshift-console/console-6db77f4495-958wf: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:14.257110 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.257063 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:14.757053799 +0000 UTC m=+1.714638756 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5mwlv" (UniqueName: "kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:14.267686 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.267666 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg65k\" (UniqueName: \"kubernetes.io/projected/efbcf64e-3ade-4343-a9c8-bf945b141a02-kube-api-access-kg65k\") pod \"network-metrics-daemon-brh9c\" (UID: \"efbcf64e-3ade-4343-a9c8-bf945b141a02\") " pod="openshift-multus/network-metrics-daemon-brh9c" Nov 04 05:29:14.267781 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.267691 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzqnm\" (UniqueName: \"kubernetes.io/projected/7239836a-64ce-4847-91f4-9360cd652ea6-kube-api-access-pzqnm\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.268261 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.268246 5904 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 04 05:29:14.268298 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.268264 5904 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 04 05:29:14.268298 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.268275 5904 projected.go:196] Error preparing data for projected volume kube-api-access-xnhgf for pod openshift-console/console-5f5f49d4fc-6cgqw: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:14.268403 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.268307 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:14.768297808 +0000 UTC m=+1.725882740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xnhgf" (UniqueName: "kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:14.269517 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.269502 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6rf9\" (UniqueName: \"kubernetes.io/projected/343f8865-3015-4715-960c-a2e1418c6f7e-kube-api-access-x6rf9\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.269559 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.269522 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc46j\" (UniqueName: \"kubernetes.io/projected/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-kube-api-access-rc46j\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.305366 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.305340 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:29:14.305366 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.305344 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 05:29:14.305524 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.305347 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 05:29:14.305524 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.305514 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Nov 04 05:29:14.306123 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.306105 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 05:29:14.306299 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.306254 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 05:29:14.306299 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.306259 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 05:29:14.306429 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.306378 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Nov 04 05:29:14.306682 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.306663 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Nov 04 05:29:14.306772 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.306702 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Nov 04 05:29:14.306772 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.306734 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Nov 04 05:29:14.306772 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.306767 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Nov 04 05:29:14.306898 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.306807 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Nov 04 05:29:14.306898 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.306818 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-qxcbb\"" Nov 04 05:29:14.307577 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.307555 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.308168 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.308569 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddw4d\" (UniqueName: \"kubernetes.io/projected/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-api-access-ddw4d\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.308615 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjz6h\" (UniqueName: \"kubernetes.io/projected/52797b3b-80b4-443e-aa1d-158430ee739e-kube-api-access-sjz6h\") pod \"monitoring-plugin-77f4f76594-4s58k\" (UID: \"52797b3b-80b4-443e-aa1d-158430ee739e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.308639 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.308839 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.308913 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28fm9\" (UniqueName: \"kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-kube-api-access-28fm9\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.309271 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dmt9\" (UniqueName: \"kubernetes.io/projected/e382c8f0-2b69-4271-8098-8328ac9b53ed-kube-api-access-9dmt9\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.309485 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c76c8\" (UniqueName: \"kubernetes.io/projected/137f70c1-c852-4c5f-823f-b8f7ed565c00-kube-api-access-c76c8\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.309523 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlrrj\" (UniqueName: \"kubernetes.io/projected/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-kube-api-access-wlrrj\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.310439 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/137f70c1-c852-4c5f-823f-b8f7ed565c00-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.310508 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qm9rq\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.310521 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.310775 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.311404 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d92sv\" (UniqueName: \"kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-kube-api-access-d92sv\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.311832 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.311864 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.311933 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qlwsz\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.312139 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.312152 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.312315 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.312325 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.312435 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.312526 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.312671 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.312689 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.312765 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.312819 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.312935 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.312986 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.312999 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.313021 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.313051 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.313122 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-chnwp\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.313163 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.313372 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ghvrd\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.313541 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.313573 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.313640 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.313744 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.313374 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.312693 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fwwlp\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.313831 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-263s2l7uf16b\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.313926 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.313960 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314017 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrnng\" (UniqueName: \"kubernetes.io/projected/32bea234-f8f1-41fa-ac2f-bea97c729d10-kube-api-access-mrnng\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314020 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314065 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314103 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314202 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-h4l2m\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314258 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-secret-metrics-server-client-certs\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314271 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314313 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314453 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2jgc2\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314518 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32bea234-f8f1-41fa-ac2f-bea97c729d10-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314557 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314569 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314616 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8w4gl\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314690 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314749 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tbvlr\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314809 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/32bea234-f8f1-41fa-ac2f-bea97c729d10-metrics-server-audit-profiles\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314844 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314897 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314944 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.314981 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.315008 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-env-overrides\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.315099 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.315166 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.315204 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.315207 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 05:29:14.315735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.315247 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.315886 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-gqfxb\"" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.315935 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.315974 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.315975 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-web-config\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.315988 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e19ce810-1d70-46db-a8ce-d824a9ffc81e-config-volume\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316026 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316000 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pwkwp\"" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316060 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316069 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-lllmb\"" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316100 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-62jkh\"" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316108 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316132 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-teufh2l7p2i9\"" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316311 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316394 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-35ajntqbki0o8\"" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316450 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/137f70c1-c852-4c5f-823f-b8f7ed565c00-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316497 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316569 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316633 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316644 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316746 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316799 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghglb\" (UniqueName: \"kubernetes.io/projected/80902534-76da-4e9d-8157-ce481e71f2f8-kube-api-access-ghglb\") pod \"node-resolver-fvxf5\" (UID: \"80902534-76da-4e9d-8157-ce481e71f2f8\") " pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316809 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-tls\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316910 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-client-ca-bundle\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.316975 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.317019 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.317182 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/82660df8-f278-4d7c-b02a-149527e30d27-ovn-node-metrics-cert\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.317331 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-metrics-client-ca\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.317390 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8wjt\" (UniqueName: \"kubernetes.io/projected/e19ce810-1d70-46db-a8ce-d824a9ffc81e-kube-api-access-x8wjt\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.317511 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7b4c728-a80c-403d-88d9-af4fb0835993-iptables-alerter-script\") pod \"iptables-alerter-m77gq\" (UID: \"f7b4c728-a80c-403d-88d9-af4fb0835993\") " pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.317651 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc4df\" (UniqueName: \"kubernetes.io/projected/f7b4c728-a80c-403d-88d9-af4fb0835993-kube-api-access-tc4df\") pod \"iptables-alerter-m77gq\" (UID: \"f7b4c728-a80c-403d-88d9-af4fb0835993\") " pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.317735 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.317847 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/137f70c1-c852-4c5f-823f-b8f7ed565c00-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-m96wd\" (UID: \"137f70c1-c852-4c5f-823f-b8f7ed565c00\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-m96wd" Nov 04 05:29:14.318030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.317992 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c93278df-81cb-480e-9151-ba3056f43d74-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:29:14.319246 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.318097 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.319246 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.318301 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt7bc\" (UniqueName: \"kubernetes.io/projected/9f075074-0903-4de4-b247-59a26f5b5f95-kube-api-access-mt7bc\") pod \"ingress-canary-2k9r5\" (UID: \"9f075074-0903-4de4-b247-59a26f5b5f95\") " pod="openshift-ingress-canary/ingress-canary-2k9r5" Nov 04 05:29:14.319246 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.318367 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.319246 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.318379 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52797b3b-80b4-443e-aa1d-158430ee739e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-4s58k\" (UID: \"52797b3b-80b4-443e-aa1d-158430ee739e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" Nov 04 05:29:14.319246 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.318391 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e382c8f0-2b69-4271-8098-8328ac9b53ed-metrics-client-ca\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.319246 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.318396 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.319246 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.318379 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.319246 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.318549 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c93278df-81cb-480e-9151-ba3056f43d74-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:29:14.319246 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.318592 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.319246 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.318686 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.319246 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.318854 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjjl9\" (UniqueName: \"kubernetes.io/projected/82660df8-f278-4d7c-b02a-149527e30d27-kube-api-access-pjjl9\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.319246 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.318992 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:29:14.319246 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.319001 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzdc9\" (UniqueName: \"kubernetes.io/projected/c93278df-81cb-480e-9151-ba3056f43d74-kube-api-access-wzdc9\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:29:14.319246 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.319126 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:29:14.320405 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.320381 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e19ce810-1d70-46db-a8ce-d824a9ffc81e-metrics-tls\") pod \"dns-default-9zfz8\" (UID: \"e19ce810-1d70-46db-a8ce-d824a9ffc81e\") " pod="openshift-dns/dns-default-9zfz8" Nov 04 05:29:14.320500 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.320428 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/82660df8-f278-4d7c-b02a-149527e30d27-ovnkube-script-lib\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.320500 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.320440 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e382c8f0-2b69-4271-8098-8328ac9b53ed-secret-grpc-tls\") pod \"thanos-querier-99867478d-zstlh\" (UID: \"e382c8f0-2b69-4271-8098-8328ac9b53ed\") " pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:14.320500 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.320471 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-tls\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.320875 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.320588 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-config-volume\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.320875 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.320594 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/32bea234-f8f1-41fa-ac2f-bea97c729d10-secret-metrics-server-tls\") pod \"metrics-server-765454fc56-b5f6j\" (UID: \"32bea234-f8f1-41fa-ac2f-bea97c729d10\") " pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:29:14.321098 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.321081 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-config\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.321219 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.321205 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.321306 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.321279 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-accelerators-collector-config\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.321356 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.321314 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.321356 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.321346 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-web-config\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.321481 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.321468 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f075074-0903-4de4-b247-59a26f5b5f95-cert\") pod \"ingress-canary-2k9r5\" (UID: \"9f075074-0903-4de4-b247-59a26f5b5f95\") " pod="openshift-ingress-canary/ingress-canary-2k9r5" Nov 04 05:29:14.321755 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.321741 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.322496 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.322470 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.322630 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.322614 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-97647\" (UID: \"ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-97647" Nov 04 05:29:14.322630 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.322620 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.324865 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.324848 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Nov 04 05:29:14.324940 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.324926 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Nov 04 05:29:14.325007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.324945 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Nov 04 05:29:14.328965 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.328945 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:14.329745 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.329725 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.330604 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.330582 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:14.353802 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.353779 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-var-lib-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.353910 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.353808 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-run\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.353910 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.353825 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-cnibin\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.353910 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.353850 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-systemd\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.353910 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.353874 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-var-lib-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.353910 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.353882 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-modprobe-d\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.353905 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-run\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.353914 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-cnibin\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.353922 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-host\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.353949 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7b4c728-a80c-403d-88d9-af4fb0835993-host-slash\") pod \"iptables-alerter-m77gq\" (UID: \"f7b4c728-a80c-403d-88d9-af4fb0835993\") " pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.353967 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-run-ovn-kubernetes\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.353967 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-modprobe-d\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.353976 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-host\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.353951 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-systemd\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.353987 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354008 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-cni-netd\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354017 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-run-ovn-kubernetes\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354025 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-etc-kubernetes\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354032 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354009 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7b4c728-a80c-403d-88d9-af4fb0835993-host-slash\") pod \"iptables-alerter-m77gq\" (UID: \"f7b4c728-a80c-403d-88d9-af4fb0835993\") " pod="openshift-network-operator/iptables-alerter-m77gq" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354043 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-system-cni-dir\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354044 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-cni-netd\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.354057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354061 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-etc-kubernetes\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354078 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-slash\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354090 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-system-cni-dir\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354108 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-kubelet\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354123 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-etc-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354138 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-systemd\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354156 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-etc-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354162 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-kubelet\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354160 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-k8s-cni-cncf-io\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354192 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-k8s-cni-cncf-io\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354176 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-slash\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354199 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-systemd\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354197 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-host\") pod \"node-ca-rkg7g\" (UID: \"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01\") " pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354264 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bcebbdb-3bcd-4050-94b5-3139d6cf3d01-host\") pod \"node-ca-rkg7g\" (UID: \"4bcebbdb-3bcd-4050-94b5-3139d6cf3d01\") " pod="openshift-image-registry/node-ca-rkg7g" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354287 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-socket-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354305 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-device-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354323 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-sys\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354317 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-socket-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354345 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/80902534-76da-4e9d-8157-ce481e71f2f8-hosts-file\") pod \"node-resolver-fvxf5\" (UID: \"80902534-76da-4e9d-8157-ce481e71f2f8\") " pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354351 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-device-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354365 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-cni-bin\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354374 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-sys\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354383 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/80902534-76da-4e9d-8157-ce481e71f2f8-hosts-file\") pod \"node-resolver-fvxf5\" (UID: \"80902534-76da-4e9d-8157-ce481e71f2f8\") " pod="openshift-dns/node-resolver-fvxf5" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354416 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-ovn\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354443 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-cni-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354444 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-cni-bin\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354475 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-ovn\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354482 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c93278df-81cb-480e-9151-ba3056f43d74-crio-socket\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354486 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-cni-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354505 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c93278df-81cb-480e-9151-ba3056f43d74-crio-socket\") pod \"insights-runtime-extractor-skkvk\" (UID: \"c93278df-81cb-480e-9151-ba3056f43d74\") " pod="openshift-insights/insights-runtime-extractor-skkvk" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354512 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-registration-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354529 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-system-cni-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354560 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-sys-fs\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354562 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-registration-dir\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354586 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-hostroot\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354621 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-system-cni-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354624 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysconfig\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.354618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354633 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-sys-fs\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354658 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/379bdc95-36d7-47f2-9e17-a96e51da6495-dbus\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354665 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysconfig\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354687 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/379bdc95-36d7-47f2-9e17-a96e51da6495-dbus\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354690 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-cni-bin\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354709 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-cni-bin\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354711 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/379bdc95-36d7-47f2-9e17-a96e51da6495-kubelet-config\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354659 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-hostroot\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354734 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/379bdc95-36d7-47f2-9e17-a96e51da6495-kubelet-config\") pod \"global-pull-secret-syncer-69rl9\" (UID: \"379bdc95-36d7-47f2-9e17-a96e51da6495\") " pod="kube-system/global-pull-secret-syncer-69rl9" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354735 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-run-netns\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354768 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-multus-certs\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354771 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-run-netns\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354802 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-systemd-units\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354818 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysctl-d\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354833 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-netns\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354838 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-multus-certs\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354860 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-cni-multus\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354864 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-systemd-units\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354877 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-run-netns\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354894 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-cni-multus\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354903 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-kubelet\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354928 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-wtmp\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354942 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysctl-d\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354945 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-conf-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354947 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-host-var-lib-kubelet\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354966 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-conf-dir\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354968 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.354991 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-node-exporter-wtmp\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355003 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-node-log\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355019 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355035 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-root\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355050 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysctl-conf\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355054 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-node-log\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355066 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355074 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355079 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-root\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355085 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-lib-modules\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355105 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-var-lib-kubelet\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355124 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-socket-dir-parent\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355149 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-cnibin\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355160 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-lib-modules\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355170 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-sysctl-conf\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355178 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-var-lib-kubelet\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355184 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7239836a-64ce-4847-91f4-9360cd652ea6-multus-socket-dir-parent\") pod \"multus-hk726\" (UID: \"7239836a-64ce-4847-91f4-9360cd652ea6\") " pod="openshift-multus/multus-hk726" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355210 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-sys\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355184 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7374bdb0-a62f-4b1c-9d14-e1097c851ff0-sys\") pod \"node-exporter-dzs66\" (UID: \"7374bdb0-a62f-4b1c-9d14-e1097c851ff0\") " pod="openshift-monitoring/node-exporter-dzs66" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355266 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-etc-selinux\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355282 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-log-socket\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355297 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-kubernetes\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355323 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355217 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4a51628-0aab-4614-a4ed-11f184521cd4-cnibin\") pod \"multus-additional-cni-plugins-z9djg\" (UID: \"f4a51628-0aab-4614-a4ed-11f184521cd4\") " pod="openshift-multus/multus-additional-cni-plugins-z9djg" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355334 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dbbe9ebe-9bf8-43fd-88ff-8b5b23126202-etc-selinux\") pod \"aws-ebs-csi-driver-node-fscnx\" (UID: \"dbbe9ebe-9bf8-43fd-88ff-8b5b23126202\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fscnx" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355346 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-log-socket\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355388 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82660df8-f278-4d7c-b02a-149527e30d27-run-openvswitch\") pod \"ovnkube-node-slrx2\" (UID: \"82660df8-f278-4d7c-b02a-149527e30d27\") " pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:14.355672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.355421 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/343f8865-3015-4715-960c-a2e1418c6f7e-etc-kubernetes\") pod \"tuned-gbms4\" (UID: \"343f8865-3015-4715-960c-a2e1418c6f7e\") " pod="openshift-cluster-node-tuning-operator/tuned-gbms4" Nov 04 05:29:14.759695 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.759658 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mwlv\" (UniqueName: \"kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:14.759864 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.759707 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:14.759864 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.759731 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:14.759864 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.759746 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:14.759864 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.759766 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:14.759864 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.759788 5904 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:14.759864 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.759833 5904 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:14.759864 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.759837 5904 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 04 05:29:14.759864 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.759795 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:14.759864 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.759862 5904 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.759877 5904 projected.go:196] Error preparing data for projected volume kube-api-access-5mwlv for pod openshift-console/console-6db77f4495-958wf: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.759863 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:15.759844934 +0000 UTC m=+2.717429879 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.759885 5904 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.759890 5904 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.759916 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:15.759891864 +0000 UTC m=+2.717476797 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.759936 5904 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.759940 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:15.759927312 +0000 UTC m=+2.717512246 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5mwlv" (UniqueName: "kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.759962 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:15.759953795 +0000 UTC m=+2.717538724 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.759979 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:15.759971065 +0000 UTC m=+2.717556007 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.759996 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:15.7599886 +0000 UTC m=+2.717573529 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-config" not registered Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.760032 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.760052 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.760087 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.760120 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.760139 5904 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.760148 5904 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.760174 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:15.7601621 +0000 UTC m=+2.717747046 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"service-ca" not registered Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.760191 5904 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 04 05:29:14.760182 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.760193 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:14.760794 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.760197 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:15.760185266 +0000 UTC m=+2.717770213 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:14.760794 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.760216 5904 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 04 05:29:14.760794 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.760272 5904 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:14.760794 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.760245 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:15.760213621 +0000 UTC m=+2.717798579 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"service-ca" not registered Nov 04 05:29:14.760794 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.760324 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:15.760309239 +0000 UTC m=+2.717894182 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-config" not registered Nov 04 05:29:14.760794 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.760351 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:15.760340427 +0000 UTC m=+2.717925361 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:14.760794 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.760403 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:14.760794 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.760434 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:14.760794 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.760483 5904 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:14.760794 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.760521 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:15.760513316 +0000 UTC m=+2.718098249 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:14.760794 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.760556 5904 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:14.760794 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.760599 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:15.760587416 +0000 UTC m=+2.718172359 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:14.861548 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:14.861508 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnhgf\" (UniqueName: \"kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:14.861722 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.861664 5904 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 04 05:29:14.861722 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.861686 5904 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 04 05:29:14.861722 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.861696 5904 projected.go:196] Error preparing data for projected volume kube-api-access-xnhgf for pod openshift-console/console-5f5f49d4fc-6cgqw: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:14.861835 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:14.861743 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:15.861730118 +0000 UTC m=+2.819315059 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnhgf" (UniqueName: "kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:15.673874 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:15.673848 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:15.772007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:15.771962 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:15.772007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:15.772011 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:15.772195 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:15.772033 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mwlv\" (UniqueName: \"kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:15.772195 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:15.772069 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:15.772195 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:15.772090 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:15.772195 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:15.772106 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:15.772195 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772121 5904 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:15.772195 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772139 5904 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 04 05:29:15.772195 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772163 5904 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 04 05:29:15.772195 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772165 5904 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:15.772195 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772176 5904 projected.go:196] Error preparing data for projected volume kube-api-access-5mwlv for pod openshift-console/console-6db77f4495-958wf: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:15.772195 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:15.772131 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:15.772195 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772198 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:17.77218018 +0000 UTC m=+4.729765132 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772163 5904 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772254 5904 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772252 5904 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772255 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:17.772215229 +0000 UTC m=+4.729800163 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772341 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:17.772323761 +0000 UTC m=+4.729908702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5mwlv" (UniqueName: "kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772365 5904 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772368 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:17.772357136 +0000 UTC m=+4.729942064 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-config" not registered Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772421 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:17.772409641 +0000 UTC m=+4.729994572 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772434 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:17.772427999 +0000 UTC m=+4.730012931 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772445 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:17.772440327 +0000 UTC m=+4.730025257 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:15.772487 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:15.772556 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772576 5904 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772604 5904 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772606 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:17.772599017 +0000 UTC m=+4.730183945 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:15.772579 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772627 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:17.772620949 +0000 UTC m=+4.730205924 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"service-ca" not registered Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:15.772675 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:15.772707 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772688 5904 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:15.773355 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:15.772727 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:15.773355 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772740 5904 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 04 05:29:15.773355 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772763 5904 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 04 05:29:15.773355 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772743 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:17.772726453 +0000 UTC m=+4.730311396 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:15.773355 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772797 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:17.772784927 +0000 UTC m=+4.730369862 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-config" not registered Nov 04 05:29:15.773355 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772811 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:17.772805632 +0000 UTC m=+4.730390559 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"service-ca" not registered Nov 04 05:29:15.773355 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:15.772852 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:15.773355 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772905 5904 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:15.773355 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.772940 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:17.772929536 +0000 UTC m=+4.730514464 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:15.873733 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:15.873691 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnhgf\" (UniqueName: \"kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:15.873892 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.873856 5904 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 04 05:29:15.873892 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.873883 5904 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 04 05:29:15.873993 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.873895 5904 projected.go:196] Error preparing data for projected volume kube-api-access-xnhgf for pod openshift-console/console-5f5f49d4fc-6cgqw: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:15.873993 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:15.873953 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:17.873935336 +0000 UTC m=+4.831520279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnhgf" (UniqueName: "kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:16.419521 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:16.419485 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9zfz8" Nov 04 05:29:16.762935 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:16.762864 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-765454fc56-b5f6j" Nov 04 05:29:17.587995 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:17.587967 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:17.588262 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:17.588248 5904 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 05:29:17.796245 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:17.796198 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:17.796269 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796310 5904 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:17.796325 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:17.796373 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796378 5904 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:17.796404 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796443 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:21.796422705 +0000 UTC m=+8.754007649 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"service-ca" not registered Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796463 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:21.796453756 +0000 UTC m=+8.754038700 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-config" not registered Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796464 5904 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:17.796485 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mwlv\" (UniqueName: \"kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796492 5904 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796503 5904 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796522 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:21.796509372 +0000 UTC m=+8.754094316 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796556 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:21.796539765 +0000 UTC m=+8.754124710 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796581 5904 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796595 5904 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796607 5904 projected.go:196] Error preparing data for projected volume kube-api-access-5mwlv for pod openshift-console/console-6db77f4495-958wf: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796612 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:21.796588058 +0000 UTC m=+8.754173001 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796632 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:21.796625377 +0000 UTC m=+8.754210350 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5mwlv" (UniqueName: "kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:17.796666 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:17.796727 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:17.796708 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796756 5904 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796780 5904 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796791 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:21.796781445 +0000 UTC m=+8.754366386 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796811 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:21.796802207 +0000 UTC m=+8.754387149 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:17.796826 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:17.796847 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:17.796896 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796918 5904 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:17.796936 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796950 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:21.7969405 +0000 UTC m=+8.754525435 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796950 5904 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.796987 5904 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:17.796983 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.797008 5904 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.797020 5904 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.797023 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:21.797008041 +0000 UTC m=+8.754592974 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-config" not registered Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.797058 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:21.797051141 +0000 UTC m=+8.754636075 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.797071 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:21.797066186 +0000 UTC m=+8.754651114 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:17.797544 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.797083 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:21.797077193 +0000 UTC m=+8.754662121 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"service-ca" not registered Nov 04 05:29:17.898132 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:17.898051 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnhgf\" (UniqueName: \"kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:17.898303 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.898203 5904 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 04 05:29:17.898303 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.898221 5904 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 04 05:29:17.898303 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.898254 5904 projected.go:196] Error preparing data for projected volume kube-api-access-xnhgf for pod openshift-console/console-5f5f49d4fc-6cgqw: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:17.898432 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:17.898304 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:21.898293061 +0000 UTC m=+8.855878009 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnhgf" (UniqueName: "kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:18.135944 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:18.135916 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-99867478d-zstlh" Nov 04 05:29:18.167413 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:18.167337 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:21.547093 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:21.547058 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:29:21.837476 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:21.837442 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:21.837621 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:21.837488 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:21.837621 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.837562 5904 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 04 05:29:21.837621 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:21.837580 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:21.837621 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.837620 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:29.837607343 +0000 UTC m=+16.795192275 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"service-ca" not registered Nov 04 05:29:21.837774 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.837634 5904 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:21.837774 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:21.837656 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:21.837774 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.837672 5904 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 04 05:29:21.837774 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.837693 5904 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:21.837774 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.837676 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:29.837663725 +0000 UTC m=+16.795248691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:21.837774 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.837734 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:29.837724847 +0000 UTC m=+16.795309786 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-config" not registered Nov 04 05:29:21.837774 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.837751 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:29.837745214 +0000 UTC m=+16.795330142 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:21.837774 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:21.837762 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:21.838036 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:21.837782 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mwlv\" (UniqueName: \"kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:21.838036 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:21.837811 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:21.838036 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:21.837842 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:21.838036 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:21.837870 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:21.838036 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.837879 5904 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:21.838036 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:21.837899 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:21.838036 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.837904 5904 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:21.838036 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.837913 5904 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 04 05:29:21.838036 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.837927 5904 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 04 05:29:21.838036 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.837935 5904 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 04 05:29:21.838036 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.837941 5904 projected.go:196] Error preparing data for projected volume kube-api-access-5mwlv for pod openshift-console/console-6db77f4495-958wf: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:21.838036 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.837962 5904 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:21.838036 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.837915 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:29.837904244 +0000 UTC m=+16.795489200 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:21.838036 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.838000 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:29.837989179 +0000 UTC m=+16.795574109 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:21.838036 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.838017 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:29.838009258 +0000 UTC m=+16.795594187 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-config" not registered Nov 04 05:29:21.838036 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.838021 5904 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:21.838549 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.838030 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:29.838024361 +0000 UTC m=+16.795609289 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5mwlv" (UniqueName: "kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:21.838549 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.838093 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:29.838082008 +0000 UTC m=+16.795666940 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:21.838549 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.838107 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:29.838101939 +0000 UTC m=+16.795686867 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:21.838549 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:21.838143 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:21.838549 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:21.838171 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:21.838549 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:21.838191 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:21.838549 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.838274 5904 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:21.838549 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.838282 5904 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 04 05:29:21.838549 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.838285 5904 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:21.838549 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.838317 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:29.83830149 +0000 UTC m=+16.795886439 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:21.838549 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.838336 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:29.838326138 +0000 UTC m=+16.795911068 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"service-ca" not registered Nov 04 05:29:21.838549 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.838352 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:29.838343546 +0000 UTC m=+16.795928483 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:21.938826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:21.938795 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnhgf\" (UniqueName: \"kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:21.938957 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.938946 5904 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 04 05:29:21.938999 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.938963 5904 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 04 05:29:21.938999 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.938973 5904 projected.go:196] Error preparing data for projected volume kube-api-access-xnhgf for pod openshift-console/console-5f5f49d4fc-6cgqw: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:21.939060 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:21.939021 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:29.939007812 +0000 UTC m=+16.896592743 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnhgf" (UniqueName: "kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:23.721174 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:23.721137 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-4s58k" Nov 04 05:29:27.132095 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:27.132021 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:29:29.923621 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:29.923585 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:29.923621 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:29.923627 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:29.923658 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:29.923691 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:29.923708 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:29.923732 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mwlv\" (UniqueName: \"kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.923734 5904 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:29.923775 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.923803 5904 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:29.923810 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.923822 5904 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.923739 5904 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.923831 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:45.923809802 +0000 UTC m=+32.881394749 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"service-ca" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.923883 5904 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.923903 5904 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.923883 5904 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.923922 5904 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.923934 5904 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.923944 5904 projected.go:196] Error preparing data for projected volume kube-api-access-5mwlv for pod openshift-console/console-6db77f4495-958wf: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.923906 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:45.923888769 +0000 UTC m=+32.881473708 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.923969 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:45.923960866 +0000 UTC m=+32.881545828 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5mwlv" (UniqueName: "kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.923986 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:45.923977815 +0000 UTC m=+32.881562744 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:29.924003 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:29.924028 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.924056 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:45.924044931 +0000 UTC m=+32.881629888 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-config" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.924070 5904 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.924078 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:45.924069278 +0000 UTC m=+32.881654205 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.924096 5904 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.924100 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:45.924092222 +0000 UTC m=+32.881677154 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"console-oauth-config" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.924183 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:45.924165221 +0000 UTC m=+32.881750164 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:29.924180 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.924207 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:45.924197741 +0000 UTC m=+32.881782669 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"oauth-serving-cert" not registered Nov 04 05:29:29.925331 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.924248 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:45.924219056 +0000 UTC m=+32.881803992 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-config" not registered Nov 04 05:29:29.925331 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:29.924298 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:29.925331 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:29.924345 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:29.925331 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.924404 5904 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:29.925331 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.924425 5904 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:29.925331 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.924446 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:45.924433658 +0000 UTC m=+32.882018608 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 04 05:29:29.925331 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.924464 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:45.924453194 +0000 UTC m=+32.882038137 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : object "openshift-console"/"console-serving-cert" not registered Nov 04 05:29:29.925331 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:29.924483 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca\") pod \"console-6db77f4495-958wf\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:29.925331 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.924584 5904 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 04 05:29:29.925331 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:29.924627 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca podName:e39328c2-e756-4c99-9d13-6fbdccae91cb nodeName:}" failed. No retries permitted until 2026-11-04 05:29:45.924615102 +0000 UTC m=+32.882200050 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca") pod "console-6db77f4495-958wf" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb") : object "openshift-console"/"service-ca" not registered Nov 04 05:29:30.025847 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:30.025814 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnhgf\" (UniqueName: \"kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf\") pod \"console-5f5f49d4fc-6cgqw\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:30.026025 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:30.025987 5904 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 04 05:29:30.026025 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:30.026010 5904 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 04 05:29:30.026025 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:30.026022 5904 projected.go:196] Error preparing data for projected volume kube-api-access-xnhgf for pod openshift-console/console-5f5f49d4fc-6cgqw: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:30.026184 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:30.026082 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf podName:7beaf096-fa6b-4233-a68b-1ab974c2e037 nodeName:}" failed. No retries permitted until 2026-11-04 05:29:46.026065201 +0000 UTC m=+32.983650145 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnhgf" (UniqueName: "kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf") pod "console-5f5f49d4fc-6cgqw" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 04 05:29:30.521969 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:30.521937 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:29:30.522320 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:30.522287 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="alertmanager" containerID="cri-o://f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7" gracePeriod=120 Nov 04 05:29:30.522418 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:30.522309 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="kube-rbac-proxy" containerID="cri-o://9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78" gracePeriod=120 Nov 04 05:29:30.522418 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:30.522320 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="prom-label-proxy" containerID="cri-o://bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25" gracePeriod=120 Nov 04 05:29:30.522418 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:30.522343 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="kube-rbac-proxy-metric" containerID="cri-o://601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b" gracePeriod=120 Nov 04 05:29:30.522418 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:30.522396 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="config-reloader" containerID="cri-o://21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78" gracePeriod=120 Nov 04 05:29:30.522643 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:30.522343 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="kube-rbac-proxy-web" containerID="cri-o://114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c" gracePeriod=120 Nov 04 05:29:31.310337 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.310301 5904 generic.go:356] "Generic (PLEG): container finished" podID="dcd56353-442a-468a-88b7-6033c8686386" containerID="bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25" exitCode=0 Nov 04 05:29:31.310337 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.310331 5904 generic.go:356] "Generic (PLEG): container finished" podID="dcd56353-442a-468a-88b7-6033c8686386" containerID="9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78" exitCode=0 Nov 04 05:29:31.310337 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.310340 5904 generic.go:356] "Generic (PLEG): container finished" podID="dcd56353-442a-468a-88b7-6033c8686386" containerID="21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78" exitCode=0 Nov 04 05:29:31.310783 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.310350 5904 generic.go:356] "Generic (PLEG): container finished" podID="dcd56353-442a-468a-88b7-6033c8686386" containerID="f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7" exitCode=0 Nov 04 05:29:31.310783 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.310363 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dcd56353-442a-468a-88b7-6033c8686386","Type":"ContainerDied","Data":"bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25"} Nov 04 05:29:31.310783 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.310433 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dcd56353-442a-468a-88b7-6033c8686386","Type":"ContainerDied","Data":"9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78"} Nov 04 05:29:31.310783 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.310460 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dcd56353-442a-468a-88b7-6033c8686386","Type":"ContainerDied","Data":"21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78"} Nov 04 05:29:31.310783 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.310485 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dcd56353-442a-468a-88b7-6033c8686386","Type":"ContainerDied","Data":"f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7"} Nov 04 05:29:31.806257 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.806218 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:31.844080 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.844015 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-tls-assets\" (UniqueName: \"kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-tls-assets\") pod \"dcd56353-442a-468a-88b7-6033c8686386\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " Nov 04 05:29:31.844080 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.844050 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-metrics-client-ca\") pod \"dcd56353-442a-468a-88b7-6033c8686386\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " Nov 04 05:29:31.844352 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.844089 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-web-config\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-web-config\") pod \"dcd56353-442a-468a-88b7-6033c8686386\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " Nov 04 05:29:31.844352 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.844146 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-kube-api-access-28fm9\" (UniqueName: \"kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-kube-api-access-28fm9\") pod \"dcd56353-442a-468a-88b7-6033c8686386\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " Nov 04 05:29:31.844352 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.844166 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-main-tls\") pod \"dcd56353-442a-468a-88b7-6033c8686386\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " Nov 04 05:29:31.844352 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.844272 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-trusted-ca-bundle\") pod \"dcd56353-442a-468a-88b7-6033c8686386\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " Nov 04 05:29:31.844352 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.844307 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-web\") pod \"dcd56353-442a-468a-88b7-6033c8686386\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " Nov 04 05:29:31.844352 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.844339 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy\") pod \"dcd56353-442a-468a-88b7-6033c8686386\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " Nov 04 05:29:31.844649 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.844370 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-config-out\" (UniqueName: \"kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-config-out\") pod \"dcd56353-442a-468a-88b7-6033c8686386\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " Nov 04 05:29:31.844776 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.844750 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-trusted-ca-bundle" pod "dcd56353-442a-468a-88b7-6033c8686386" (UID: "dcd56353-442a-468a-88b7-6033c8686386"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:31.844851 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.844824 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-metric\") pod \"dcd56353-442a-468a-88b7-6033c8686386\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " Nov 04 05:29:31.844903 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.844880 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-cluster-tls-config\") pod \"dcd56353-442a-468a-88b7-6033c8686386\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " Nov 04 05:29:31.844953 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.844916 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-main-db\") pod \"dcd56353-442a-468a-88b7-6033c8686386\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " Nov 04 05:29:31.845002 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.844973 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-config-volume\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-config-volume\") pod \"dcd56353-442a-468a-88b7-6033c8686386\" (UID: \"dcd56353-442a-468a-88b7-6033c8686386\") " Nov 04 05:29:31.845344 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.845326 5904 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:31.846858 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.846617 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-metrics-client-ca" pod "dcd56353-442a-468a-88b7-6033c8686386" (UID: "dcd56353-442a-468a-88b7-6033c8686386"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:31.847546 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.847493 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-main-db" pod "dcd56353-442a-468a-88b7-6033c8686386" (UID: "dcd56353-442a-468a-88b7-6033c8686386"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:29:31.847945 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.847898 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-tls-assets" pod "dcd56353-442a-468a-88b7-6033c8686386" (UID: "dcd56353-442a-468a-88b7-6033c8686386"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:29:31.852618 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.852592 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-web" pod "dcd56353-442a-468a-88b7-6033c8686386" (UID: "dcd56353-442a-468a-88b7-6033c8686386"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:31.852721 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.852696 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-config-out" pod "dcd56353-442a-468a-88b7-6033c8686386" (UID: "dcd56353-442a-468a-88b7-6033c8686386"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:29:31.852801 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.852713 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy" pod "dcd56353-442a-468a-88b7-6033c8686386" (UID: "dcd56353-442a-468a-88b7-6033c8686386"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:31.852855 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.852839 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-metric" pod "dcd56353-442a-468a-88b7-6033c8686386" (UID: "dcd56353-442a-468a-88b7-6033c8686386"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:31.852954 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.852935 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-kube-api-access-28fm9" pod "dcd56353-442a-468a-88b7-6033c8686386" (UID: "dcd56353-442a-468a-88b7-6033c8686386"). InnerVolumeSpecName "kube-api-access-28fm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:29:31.853006 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.852948 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-config-volume" pod "dcd56353-442a-468a-88b7-6033c8686386" (UID: "dcd56353-442a-468a-88b7-6033c8686386"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:31.853078 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.853062 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-main-tls" pod "dcd56353-442a-468a-88b7-6033c8686386" (UID: "dcd56353-442a-468a-88b7-6033c8686386"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:31.858005 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.857981 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-cluster-tls-config" pod "dcd56353-442a-468a-88b7-6033c8686386" (UID: "dcd56353-442a-468a-88b7-6033c8686386"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:31.860829 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.860807 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-web-config" pod "dcd56353-442a-468a-88b7-6033c8686386" (UID: "dcd56353-442a-468a-88b7-6033c8686386"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:31.946145 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.946106 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28fm9\" (UniqueName: \"kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-kube-api-access-28fm9\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:31.946145 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.946138 5904 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-main-tls\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:31.946145 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.946149 5904 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:31.946406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.946159 5904 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:31.946406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.946169 5904 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-config-out\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:31.946406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.946178 5904 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:31.946406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.946188 5904 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-cluster-tls-config\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:31.946406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.946197 5904 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dcd56353-442a-468a-88b7-6033c8686386-alertmanager-main-db\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:31.946406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.946206 5904 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-config-volume\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:31.946406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.946215 5904 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dcd56353-442a-468a-88b7-6033c8686386-tls-assets\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:31.946406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.946247 5904 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcd56353-442a-468a-88b7-6033c8686386-metrics-client-ca\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:31.946406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:31.946257 5904 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dcd56353-442a-468a-88b7-6033c8686386-web-config\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:32.317734 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.317699 5904 generic.go:356] "Generic (PLEG): container finished" podID="dcd56353-442a-468a-88b7-6033c8686386" containerID="601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b" exitCode=0 Nov 04 05:29:32.317734 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.317732 5904 generic.go:356] "Generic (PLEG): container finished" podID="dcd56353-442a-468a-88b7-6033c8686386" containerID="114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c" exitCode=0 Nov 04 05:29:32.318156 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.317793 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.318156 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.317786 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dcd56353-442a-468a-88b7-6033c8686386","Type":"ContainerDied","Data":"601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b"} Nov 04 05:29:32.318156 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.317938 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dcd56353-442a-468a-88b7-6033c8686386","Type":"ContainerDied","Data":"114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c"} Nov 04 05:29:32.318156 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.317972 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dcd56353-442a-468a-88b7-6033c8686386","Type":"ContainerDied","Data":"9c8898951746b209c4c5d3ff9c217388d6a6b5bbbed8bb14d7ff0e46c6c609be"} Nov 04 05:29:32.318156 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.318005 5904 scope.go:122] "RemoveContainer" containerID="bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25" Nov 04 05:29:32.326953 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.326933 5904 scope.go:122] "RemoveContainer" containerID="601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b" Nov 04 05:29:32.337933 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.337914 5904 scope.go:122] "RemoveContainer" containerID="9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78" Nov 04 05:29:32.345121 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.345076 5904 scope.go:122] "RemoveContainer" containerID="114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c" Nov 04 05:29:32.345248 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.345119 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:29:32.351169 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.351147 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:29:32.353485 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.353467 5904 scope.go:122] "RemoveContainer" containerID="21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78" Nov 04 05:29:32.360462 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.360444 5904 scope.go:122] "RemoveContainer" containerID="f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7" Nov 04 05:29:32.367355 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.367332 5904 scope.go:122] "RemoveContainer" containerID="b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86" Nov 04 05:29:32.373920 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.373902 5904 scope.go:122] "RemoveContainer" containerID="bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25" Nov 04 05:29:32.374478 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:32.374220 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25\": container with ID starting with bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25 not found: ID does not exist" containerID="bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25" Nov 04 05:29:32.374478 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.374283 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25"} err="failed to get container status \"bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25\": rpc error: code = NotFound desc = could not find container \"bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25\": container with ID starting with bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25 not found: ID does not exist" Nov 04 05:29:32.374478 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.374349 5904 scope.go:122] "RemoveContainer" containerID="601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b" Nov 04 05:29:32.375428 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:32.374783 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b\": container with ID starting with 601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b not found: ID does not exist" containerID="601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b" Nov 04 05:29:32.375428 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.374840 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b"} err="failed to get container status \"601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b\": rpc error: code = NotFound desc = could not find container \"601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b\": container with ID starting with 601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b not found: ID does not exist" Nov 04 05:29:32.375428 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.374860 5904 scope.go:122] "RemoveContainer" containerID="9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78" Nov 04 05:29:32.375535 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:32.375221 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78\": container with ID starting with 9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78 not found: ID does not exist" containerID="9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78" Nov 04 05:29:32.375535 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.375492 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78"} err="failed to get container status \"9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78\": rpc error: code = NotFound desc = could not find container \"9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78\": container with ID starting with 9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78 not found: ID does not exist" Nov 04 05:29:32.375535 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.375510 5904 scope.go:122] "RemoveContainer" containerID="114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c" Nov 04 05:29:32.375637 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.375565 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:29:32.375787 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:32.375772 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c\": container with ID starting with 114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c not found: ID does not exist" containerID="114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c" Nov 04 05:29:32.375830 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.375793 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c"} err="failed to get container status \"114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c\": rpc error: code = NotFound desc = could not find container \"114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c\": container with ID starting with 114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c not found: ID does not exist" Nov 04 05:29:32.375830 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.375806 5904 scope.go:122] "RemoveContainer" containerID="21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78" Nov 04 05:29:32.375924 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.375908 5904 cpu_manager.go:414] "containerMap: removing container" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="kube-rbac-proxy" Nov 04 05:29:32.375976 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.375927 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="kube-rbac-proxy" Nov 04 05:29:32.375976 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.375942 5904 cpu_manager.go:414] "containerMap: removing container" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="kube-rbac-proxy-web" Nov 04 05:29:32.375976 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.375951 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="kube-rbac-proxy-web" Nov 04 05:29:32.375976 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.375961 5904 cpu_manager.go:414] "containerMap: removing container" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="prom-label-proxy" Nov 04 05:29:32.375976 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.375966 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="prom-label-proxy" Nov 04 05:29:32.375976 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.375973 5904 cpu_manager.go:414] "containerMap: removing container" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="init-config-reloader" Nov 04 05:29:32.375976 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.375977 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="init-config-reloader" Nov 04 05:29:32.376214 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.375996 5904 cpu_manager.go:414] "containerMap: removing container" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="config-reloader" Nov 04 05:29:32.376214 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376001 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="config-reloader" Nov 04 05:29:32.376214 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376016 5904 cpu_manager.go:414] "containerMap: removing container" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="kube-rbac-proxy-metric" Nov 04 05:29:32.376214 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376022 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="kube-rbac-proxy-metric" Nov 04 05:29:32.376214 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376034 5904 cpu_manager.go:414] "containerMap: removing container" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="alertmanager" Nov 04 05:29:32.376214 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376042 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="alertmanager" Nov 04 05:29:32.376214 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:32.376053 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78\": container with ID starting with 21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78 not found: ID does not exist" containerID="21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78" Nov 04 05:29:32.376214 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376069 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78"} err="failed to get container status \"21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78\": rpc error: code = NotFound desc = could not find container \"21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78\": container with ID starting with 21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78 not found: ID does not exist" Nov 04 05:29:32.376214 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376081 5904 scope.go:122] "RemoveContainer" containerID="f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7" Nov 04 05:29:32.376214 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376131 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="kube-rbac-proxy-web" Nov 04 05:29:32.376214 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376141 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="prom-label-proxy" Nov 04 05:29:32.376214 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376146 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="init-config-reloader" Nov 04 05:29:32.376214 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376155 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="config-reloader" Nov 04 05:29:32.376214 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376161 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="kube-rbac-proxy-metric" Nov 04 05:29:32.376214 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376169 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="alertmanager" Nov 04 05:29:32.376214 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376177 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcd56353-442a-468a-88b7-6033c8686386" containerName="kube-rbac-proxy" Nov 04 05:29:32.376847 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:32.376315 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7\": container with ID starting with f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7 not found: ID does not exist" containerID="f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7" Nov 04 05:29:32.376847 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376338 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7"} err="failed to get container status \"f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7\": rpc error: code = NotFound desc = could not find container \"f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7\": container with ID starting with f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7 not found: ID does not exist" Nov 04 05:29:32.376847 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376353 5904 scope.go:122] "RemoveContainer" containerID="b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86" Nov 04 05:29:32.376847 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:32.376603 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86\": container with ID starting with b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86 not found: ID does not exist" containerID="b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86" Nov 04 05:29:32.376847 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376632 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86"} err="failed to get container status \"b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86\": rpc error: code = NotFound desc = could not find container \"b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86\": container with ID starting with b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86 not found: ID does not exist" Nov 04 05:29:32.376847 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376651 5904 scope.go:122] "RemoveContainer" containerID="bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25" Nov 04 05:29:32.377115 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376921 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25"} err="failed to get container status \"bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25\": rpc error: code = NotFound desc = could not find container \"bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25\": container with ID starting with bdd69165638769da2113b029c64ecd065cd968ebed57668bd5fb359a18793a25 not found: ID does not exist" Nov 04 05:29:32.377115 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.376941 5904 scope.go:122] "RemoveContainer" containerID="601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b" Nov 04 05:29:32.377213 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.377170 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b"} err="failed to get container status \"601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b\": rpc error: code = NotFound desc = could not find container \"601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b\": container with ID starting with 601aa414a5a264aafe94f860505d401edbaa06b6b75e9fc60eb860ca0776347b not found: ID does not exist" Nov 04 05:29:32.377213 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.377190 5904 scope.go:122] "RemoveContainer" containerID="9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78" Nov 04 05:29:32.377420 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.377399 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78"} err="failed to get container status \"9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78\": rpc error: code = NotFound desc = could not find container \"9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78\": container with ID starting with 9a7d225c717762d4bdfacabea4d9d8942cf00016907635f71e2a94dd63f0fd78 not found: ID does not exist" Nov 04 05:29:32.377492 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.377421 5904 scope.go:122] "RemoveContainer" containerID="114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c" Nov 04 05:29:32.377667 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.377647 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c"} err="failed to get container status \"114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c\": rpc error: code = NotFound desc = could not find container \"114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c\": container with ID starting with 114cbf6f2e80f3ea545bacc133f726cd646fc24ca278a7944bbc73bdf8fbab3c not found: ID does not exist" Nov 04 05:29:32.377714 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.377668 5904 scope.go:122] "RemoveContainer" containerID="21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78" Nov 04 05:29:32.377905 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.377887 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78"} err="failed to get container status \"21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78\": rpc error: code = NotFound desc = could not find container \"21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78\": container with ID starting with 21e98a63874f51e94e68d8af85ed94f4392e0cfad351e6c5fe1635d3e37adf78 not found: ID does not exist" Nov 04 05:29:32.377982 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.377907 5904 scope.go:122] "RemoveContainer" containerID="f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7" Nov 04 05:29:32.378125 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.378108 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7"} err="failed to get container status \"f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7\": rpc error: code = NotFound desc = could not find container \"f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7\": container with ID starting with f6eaef85e4fd0d98a66e4e0bfc3c6c2dd96908bafc64f5a75fed73b50da185a7 not found: ID does not exist" Nov 04 05:29:32.378170 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.378126 5904 scope.go:122] "RemoveContainer" containerID="b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86" Nov 04 05:29:32.378352 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.378334 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86"} err="failed to get container status \"b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86\": rpc error: code = NotFound desc = could not find container \"b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86\": container with ID starting with b6b592d4020ecb1f2bef2e4c162b1cdce4b0e52b7516ed3e99e244a0adeede86 not found: ID does not exist" Nov 04 05:29:32.403326 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.403306 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:29:32.403475 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.403456 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.408315 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.408294 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Nov 04 05:29:32.408433 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.408344 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Nov 04 05:29:32.408433 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.408328 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Nov 04 05:29:32.408433 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.408344 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Nov 04 05:29:32.408433 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.408377 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Nov 04 05:29:32.408433 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.408378 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Nov 04 05:29:32.408433 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.408344 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Nov 04 05:29:32.408952 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.408933 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tbvlr\"" Nov 04 05:29:32.409103 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.408961 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Nov 04 05:29:32.413538 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.413517 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Nov 04 05:29:32.450960 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.450933 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.451109 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.450974 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.451109 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.451005 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.451206 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.451186 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/215ea51a-5f86-4248-8f04-4974bf428872-tls-assets\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.451265 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.451252 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.451390 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.451372 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh5kj\" (UniqueName: \"kubernetes.io/projected/215ea51a-5f86-4248-8f04-4974bf428872-kube-api-access-hh5kj\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.451432 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.451413 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/215ea51a-5f86-4248-8f04-4974bf428872-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.451540 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.451518 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/215ea51a-5f86-4248-8f04-4974bf428872-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.451614 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.451599 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-config-volume\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.451704 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.451684 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.451770 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.451754 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/215ea51a-5f86-4248-8f04-4974bf428872-config-out\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.451825 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.451781 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-web-config\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.451825 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.451812 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/215ea51a-5f86-4248-8f04-4974bf428872-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.552376 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.552335 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/215ea51a-5f86-4248-8f04-4974bf428872-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.552376 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.552382 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/215ea51a-5f86-4248-8f04-4974bf428872-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.552623 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.552529 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-config-volume\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.552623 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.552561 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.552623 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.552611 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/215ea51a-5f86-4248-8f04-4974bf428872-config-out\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.553062 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.552638 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-web-config\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.553062 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.552665 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/215ea51a-5f86-4248-8f04-4974bf428872-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.553062 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.552766 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.553062 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.552805 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/215ea51a-5f86-4248-8f04-4974bf428872-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.553062 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.552830 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.553062 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.552876 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.553062 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.552929 5904 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 05:29:32.553062 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.552973 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/215ea51a-5f86-4248-8f04-4974bf428872-tls-assets\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.553513 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.553283 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/215ea51a-5f86-4248-8f04-4974bf428872-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.553587 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.553568 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/215ea51a-5f86-4248-8f04-4974bf428872-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.553748 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.553715 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.553812 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.553799 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh5kj\" (UniqueName: \"kubernetes.io/projected/215ea51a-5f86-4248-8f04-4974bf428872-kube-api-access-hh5kj\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.555909 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.555821 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-config-volume\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.555909 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.555837 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/215ea51a-5f86-4248-8f04-4974bf428872-config-out\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.555909 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.555860 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.556114 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.556034 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.556114 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.556060 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.556114 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.556034 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/215ea51a-5f86-4248-8f04-4974bf428872-tls-assets\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.556569 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.556552 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.556939 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.556917 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.557674 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.557657 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/215ea51a-5f86-4248-8f04-4974bf428872-web-config\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.561601 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.561581 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh5kj\" (UniqueName: \"kubernetes.io/projected/215ea51a-5f86-4248-8f04-4974bf428872-kube-api-access-hh5kj\") pod \"alertmanager-main-0\" (UID: \"215ea51a-5f86-4248-8f04-4974bf428872\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:32.814053 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:32.814015 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:29:33.110541 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:33.110508 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:29:33.113653 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:33.113628 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod215ea51a_5f86_4248_8f04_4974bf428872.slice/crio-893727b16ef7eb7a03d209d0f19894a0f030c6d381cd963c660391078231b3b4 WatchSource:0}: Error finding container 893727b16ef7eb7a03d209d0f19894a0f030c6d381cd963c660391078231b3b4: Status 404 returned error can't find the container with id 893727b16ef7eb7a03d209d0f19894a0f030c6d381cd963c660391078231b3b4 Nov 04 05:29:33.166212 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:33.166179 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dcd56353-442a-468a-88b7-6033c8686386" path="/var/lib/kubelet/pods/dcd56353-442a-468a-88b7-6033c8686386/volumes" Nov 04 05:29:33.322910 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:33.322871 5904 generic.go:356] "Generic (PLEG): container finished" podID="215ea51a-5f86-4248-8f04-4974bf428872" containerID="15e9a3733e0e02b601c293d8537882ba318976614b92ae582fac53dbb9c0d015" exitCode=0 Nov 04 05:29:33.323363 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:33.322959 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"215ea51a-5f86-4248-8f04-4974bf428872","Type":"ContainerDied","Data":"15e9a3733e0e02b601c293d8537882ba318976614b92ae582fac53dbb9c0d015"} Nov 04 05:29:33.323363 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:33.323028 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"215ea51a-5f86-4248-8f04-4974bf428872","Type":"ContainerStarted","Data":"893727b16ef7eb7a03d209d0f19894a0f030c6d381cd963c660391078231b3b4"} Nov 04 05:29:34.331273 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:34.331206 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"215ea51a-5f86-4248-8f04-4974bf428872","Type":"ContainerStarted","Data":"5f6dacf55a15c869ddf33a920079d4d540a7dab158315447d3c86ad74eb6ab42"} Nov 04 05:29:34.331684 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:34.331353 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"215ea51a-5f86-4248-8f04-4974bf428872","Type":"ContainerStarted","Data":"1522fece6fae69e6105fbd228be50524941c5cf0003beb6c29959404b8de9914"} Nov 04 05:29:34.331684 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:34.331404 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"215ea51a-5f86-4248-8f04-4974bf428872","Type":"ContainerStarted","Data":"e9d440bacedd0a8da5022b5fe6387dcaed3c1a903a8451b14748593bf723c3bb"} Nov 04 05:29:34.331684 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:34.331442 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"215ea51a-5f86-4248-8f04-4974bf428872","Type":"ContainerStarted","Data":"c8908c886861c39d9e6bf1086b13c016bd18ba10078e5d3cac3660b5c1db2c3e"} Nov 04 05:29:34.331684 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:34.331485 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"215ea51a-5f86-4248-8f04-4974bf428872","Type":"ContainerStarted","Data":"24832be94c99398b9d5efbfebf8a559bc80c18628547cf892ec3ba3153e2b1e8"} Nov 04 05:29:34.331684 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:34.331519 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"215ea51a-5f86-4248-8f04-4974bf428872","Type":"ContainerStarted","Data":"0152b52ba71c4a962e3a74f8d513437c28ee98d53429700c83a388082bfa516e"} Nov 04 05:29:34.360506 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:34.360364 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.34998604 podStartE2EDuration="2.360345215s" totalImagesPullingTime="0s" totalInitContainerRuntime="10.359175ms" isStatefulPod=false podCreationTimestamp="2026-11-04 05:29:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:29:34.358480178 +0000 UTC m=+21.316065128" watchObservedRunningTime="2026-11-04 05:29:34.360345215 +0000 UTC m=+21.317930163" Nov 04 05:29:34.782612 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:34.782504 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:29:34.782820 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:34.782792 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="prometheus" containerID="cri-o://fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b" gracePeriod=600 Nov 04 05:29:34.782820 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:34.782792 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="kube-rbac-proxy" containerID="cri-o://2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df" gracePeriod=600 Nov 04 05:29:34.782976 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:34.782822 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="thanos-sidecar" containerID="cri-o://021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff" gracePeriod=600 Nov 04 05:29:34.782976 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:34.782844 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="kube-rbac-proxy-web" containerID="cri-o://33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977" gracePeriod=600 Nov 04 05:29:34.782976 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:34.782866 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6" gracePeriod=600 Nov 04 05:29:34.782976 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:34.782899 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="config-reloader" containerID="cri-o://522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45" gracePeriod=600 Nov 04 05:29:35.055080 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.055016 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.187088 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187055 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187271 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187100 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-db\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187271 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187131 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-config-out\" (UniqueName: \"kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-config-out\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187271 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187159 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-grpc-tls\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187271 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187182 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-1\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187271 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187207 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-metrics-client-certs\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187271 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187253 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-kube-api-access-d92sv\" (UniqueName: \"kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-kube-api-access-d92sv\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187686 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187275 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-kube-rbac-proxy\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187686 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187302 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-config\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-config\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187686 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187345 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-trusted-ca-bundle\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187686 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187388 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-web-config\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-web-config\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187686 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187438 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187686 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187489 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-serving-certs-ca-bundle\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187686 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187543 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-thanos-prometheus-http-client-file\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187686 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187565 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-tls-assets\" (UniqueName: \"kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-tls-assets\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187686 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187592 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-0\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187686 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187615 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-metrics-client-ca\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187686 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187632 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-tls\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187686 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187627 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-1" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:35.187686 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187658 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-kubelet-serving-ca-bundle\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.187686 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.187677 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-2\") pod \"e402f928-dd6c-4f77-acbb-94b57e6b5188\" (UID: \"e402f928-dd6c-4f77-acbb-94b57e6b5188\") " Nov 04 05:29:35.188384 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.188065 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-serving-certs-ca-bundle" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:35.188448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.188391 5904 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.189154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.189127 5904 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.189154 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.188561 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-db" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:29:35.189354 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.188576 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-trusted-ca-bundle" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:35.189354 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.189190 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-2" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:35.190047 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.190021 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-metrics-client-ca" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:35.190584 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.190524 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-kubelet-serving-ca-bundle" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:35.191181 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.191009 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-kube-rbac-proxy" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:35.191181 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.191146 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-0" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:35.191181 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.191159 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-config-out" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:29:35.191420 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.191321 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-config" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:35.191544 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.191519 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-metrics-client-certs" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:35.191621 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.191600 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-kube-rbac-proxy-web" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:35.191730 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.191683 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-thanos-prometheus-http-client-file" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:35.191859 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.191834 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-grpc-tls" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:35.191921 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.191883 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-kube-api-access-d92sv" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "kube-api-access-d92sv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:29:35.192108 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.192085 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-thanos-sidecar-tls" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:35.192943 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.192924 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-tls-assets" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:29:35.193083 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.193066 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-tls" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:35.202615 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.202591 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-web-config" pod "e402f928-dd6c-4f77-acbb-94b57e6b5188" (UID: "e402f928-dd6c-4f77-acbb-94b57e6b5188"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:35.290315 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290284 5904 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-metrics-client-ca\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290315 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290313 5904 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290474 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290324 5904 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290474 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290335 5904 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290474 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290345 5904 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290474 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290355 5904 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-db\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290474 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290364 5904 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e402f928-dd6c-4f77-acbb-94b57e6b5188-config-out\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290474 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290373 5904 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-grpc-tls\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290474 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290381 5904 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-metrics-client-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290474 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290390 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d92sv\" (UniqueName: \"kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-kube-api-access-d92sv\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290474 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290399 5904 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-kube-rbac-proxy\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290474 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290409 5904 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-config\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290474 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290417 5904 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290474 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290426 5904 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-web-config\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290474 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290436 5904 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290474 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290445 5904 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e402f928-dd6c-4f77-acbb-94b57e6b5188-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290474 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290454 5904 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e402f928-dd6c-4f77-acbb-94b57e6b5188-tls-assets\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.290474 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.290462 5904 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e402f928-dd6c-4f77-acbb-94b57e6b5188-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:35.327791 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.327761 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m66r7" Nov 04 05:29:35.338607 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.338581 5904 generic.go:356] "Generic (PLEG): container finished" podID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerID="7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6" exitCode=0 Nov 04 05:29:35.338607 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.338603 5904 generic.go:356] "Generic (PLEG): container finished" podID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerID="2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df" exitCode=0 Nov 04 05:29:35.338607 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.338612 5904 generic.go:356] "Generic (PLEG): container finished" podID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerID="33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977" exitCode=0 Nov 04 05:29:35.339039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.338618 5904 generic.go:356] "Generic (PLEG): container finished" podID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerID="021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff" exitCode=0 Nov 04 05:29:35.339039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.338624 5904 generic.go:356] "Generic (PLEG): container finished" podID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerID="522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45" exitCode=0 Nov 04 05:29:35.339039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.338629 5904 generic.go:356] "Generic (PLEG): container finished" podID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerID="fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b" exitCode=0 Nov 04 05:29:35.339039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.338640 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e402f928-dd6c-4f77-acbb-94b57e6b5188","Type":"ContainerDied","Data":"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6"} Nov 04 05:29:35.339039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.338675 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.339039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.338705 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e402f928-dd6c-4f77-acbb-94b57e6b5188","Type":"ContainerDied","Data":"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df"} Nov 04 05:29:35.339039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.338734 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e402f928-dd6c-4f77-acbb-94b57e6b5188","Type":"ContainerDied","Data":"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977"} Nov 04 05:29:35.339039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.338759 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e402f928-dd6c-4f77-acbb-94b57e6b5188","Type":"ContainerDied","Data":"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff"} Nov 04 05:29:35.339039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.338782 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e402f928-dd6c-4f77-acbb-94b57e6b5188","Type":"ContainerDied","Data":"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45"} Nov 04 05:29:35.339039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.338782 5904 scope.go:122] "RemoveContainer" containerID="7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6" Nov 04 05:29:35.339039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.338807 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e402f928-dd6c-4f77-acbb-94b57e6b5188","Type":"ContainerDied","Data":"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b"} Nov 04 05:29:35.339039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.338831 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e402f928-dd6c-4f77-acbb-94b57e6b5188","Type":"ContainerDied","Data":"75fd2d220aed24cc4be18d597bc21fe0ae142ab309de07a195021252a7217aa6"} Nov 04 05:29:35.351372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.351352 5904 scope.go:122] "RemoveContainer" containerID="2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df" Nov 04 05:29:35.358123 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.358103 5904 scope.go:122] "RemoveContainer" containerID="33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977" Nov 04 05:29:35.362157 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.362136 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:29:35.366049 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.366025 5904 scope.go:122] "RemoveContainer" containerID="021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff" Nov 04 05:29:35.368415 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.368392 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:29:35.373390 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.373372 5904 scope.go:122] "RemoveContainer" containerID="522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45" Nov 04 05:29:35.379813 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.379794 5904 scope.go:122] "RemoveContainer" containerID="fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b" Nov 04 05:29:35.388136 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.388117 5904 scope.go:122] "RemoveContainer" containerID="3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb" Nov 04 05:29:35.392519 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392497 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:29:35.392796 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392781 5904 cpu_manager.go:414] "containerMap: removing container" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="config-reloader" Nov 04 05:29:35.392796 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392796 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="config-reloader" Nov 04 05:29:35.392917 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392809 5904 cpu_manager.go:414] "containerMap: removing container" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="kube-rbac-proxy" Nov 04 05:29:35.392917 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392814 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="kube-rbac-proxy" Nov 04 05:29:35.392917 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392824 5904 cpu_manager.go:414] "containerMap: removing container" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="init-config-reloader" Nov 04 05:29:35.392917 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392832 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="init-config-reloader" Nov 04 05:29:35.392917 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392846 5904 cpu_manager.go:414] "containerMap: removing container" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="kube-rbac-proxy-web" Nov 04 05:29:35.392917 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392854 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="kube-rbac-proxy-web" Nov 04 05:29:35.392917 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392870 5904 cpu_manager.go:414] "containerMap: removing container" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="prometheus" Nov 04 05:29:35.392917 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392877 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="prometheus" Nov 04 05:29:35.392917 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392898 5904 cpu_manager.go:414] "containerMap: removing container" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="kube-rbac-proxy-thanos" Nov 04 05:29:35.392917 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392903 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="kube-rbac-proxy-thanos" Nov 04 05:29:35.392917 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392910 5904 cpu_manager.go:414] "containerMap: removing container" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="thanos-sidecar" Nov 04 05:29:35.392917 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392915 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="thanos-sidecar" Nov 04 05:29:35.393312 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392970 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="prometheus" Nov 04 05:29:35.393312 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392981 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="kube-rbac-proxy-thanos" Nov 04 05:29:35.393312 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392988 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="thanos-sidecar" Nov 04 05:29:35.393312 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.392994 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="config-reloader" Nov 04 05:29:35.393312 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.393000 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="kube-rbac-proxy" Nov 04 05:29:35.393312 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.393005 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="init-config-reloader" Nov 04 05:29:35.393312 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.393011 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" containerName="kube-rbac-proxy-web" Nov 04 05:29:35.396036 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.396021 5904 scope.go:122] "RemoveContainer" containerID="7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6" Nov 04 05:29:35.396311 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:35.396290 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6\": container with ID starting with 7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6 not found: ID does not exist" containerID="7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6" Nov 04 05:29:35.396382 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.396323 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6"} err="failed to get container status \"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6\": rpc error: code = NotFound desc = could not find container \"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6\": container with ID starting with 7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6 not found: ID does not exist" Nov 04 05:29:35.396382 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.396346 5904 scope.go:122] "RemoveContainer" containerID="2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df" Nov 04 05:29:35.396596 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:35.396579 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df\": container with ID starting with 2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df not found: ID does not exist" containerID="2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df" Nov 04 05:29:35.396639 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.396603 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df"} err="failed to get container status \"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df\": rpc error: code = NotFound desc = could not find container \"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df\": container with ID starting with 2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df not found: ID does not exist" Nov 04 05:29:35.396639 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.396619 5904 scope.go:122] "RemoveContainer" containerID="33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977" Nov 04 05:29:35.396835 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:35.396822 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977\": container with ID starting with 33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977 not found: ID does not exist" containerID="33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977" Nov 04 05:29:35.396879 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.396838 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977"} err="failed to get container status \"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977\": rpc error: code = NotFound desc = could not find container \"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977\": container with ID starting with 33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977 not found: ID does not exist" Nov 04 05:29:35.396879 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.396849 5904 scope.go:122] "RemoveContainer" containerID="021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff" Nov 04 05:29:35.397080 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:35.397062 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff\": container with ID starting with 021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff not found: ID does not exist" containerID="021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff" Nov 04 05:29:35.397140 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.397088 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff"} err="failed to get container status \"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff\": rpc error: code = NotFound desc = could not find container \"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff\": container with ID starting with 021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff not found: ID does not exist" Nov 04 05:29:35.397140 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.397110 5904 scope.go:122] "RemoveContainer" containerID="522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45" Nov 04 05:29:35.397356 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:35.397339 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45\": container with ID starting with 522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45 not found: ID does not exist" containerID="522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45" Nov 04 05:29:35.397392 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.397360 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45"} err="failed to get container status \"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45\": rpc error: code = NotFound desc = could not find container \"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45\": container with ID starting with 522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45 not found: ID does not exist" Nov 04 05:29:35.397392 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.397374 5904 scope.go:122] "RemoveContainer" containerID="fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b" Nov 04 05:29:35.397580 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:35.397564 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b\": container with ID starting with fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b not found: ID does not exist" containerID="fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b" Nov 04 05:29:35.397636 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.397587 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b"} err="failed to get container status \"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b\": rpc error: code = NotFound desc = could not find container \"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b\": container with ID starting with fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b not found: ID does not exist" Nov 04 05:29:35.397636 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.397605 5904 scope.go:122] "RemoveContainer" containerID="3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb" Nov 04 05:29:35.397804 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:35.397790 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb\": container with ID starting with 3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb not found: ID does not exist" containerID="3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb" Nov 04 05:29:35.397837 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.397806 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb"} err="failed to get container status \"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb\": rpc error: code = NotFound desc = could not find container \"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb\": container with ID starting with 3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb not found: ID does not exist" Nov 04 05:29:35.397837 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.397818 5904 scope.go:122] "RemoveContainer" containerID="7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6" Nov 04 05:29:35.398045 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.398026 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6"} err="failed to get container status \"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6\": rpc error: code = NotFound desc = could not find container \"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6\": container with ID starting with 7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6 not found: ID does not exist" Nov 04 05:29:35.398135 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.398047 5904 scope.go:122] "RemoveContainer" containerID="2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df" Nov 04 05:29:35.398259 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.398243 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df"} err="failed to get container status \"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df\": rpc error: code = NotFound desc = could not find container \"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df\": container with ID starting with 2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df not found: ID does not exist" Nov 04 05:29:35.398311 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.398258 5904 scope.go:122] "RemoveContainer" containerID="33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977" Nov 04 05:29:35.398458 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.398440 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977"} err="failed to get container status \"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977\": rpc error: code = NotFound desc = could not find container \"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977\": container with ID starting with 33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977 not found: ID does not exist" Nov 04 05:29:35.398512 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.398459 5904 scope.go:122] "RemoveContainer" containerID="021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff" Nov 04 05:29:35.398664 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.398645 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff"} err="failed to get container status \"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff\": rpc error: code = NotFound desc = could not find container \"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff\": container with ID starting with 021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff not found: ID does not exist" Nov 04 05:29:35.398731 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.398665 5904 scope.go:122] "RemoveContainer" containerID="522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45" Nov 04 05:29:35.398885 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.398869 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45"} err="failed to get container status \"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45\": rpc error: code = NotFound desc = could not find container \"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45\": container with ID starting with 522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45 not found: ID does not exist" Nov 04 05:29:35.398929 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.398885 5904 scope.go:122] "RemoveContainer" containerID="fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b" Nov 04 05:29:35.399072 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.399057 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b"} err="failed to get container status \"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b\": rpc error: code = NotFound desc = could not find container \"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b\": container with ID starting with fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b not found: ID does not exist" Nov 04 05:29:35.399114 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.399072 5904 scope.go:122] "RemoveContainer" containerID="3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb" Nov 04 05:29:35.399278 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.399258 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb"} err="failed to get container status \"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb\": rpc error: code = NotFound desc = could not find container \"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb\": container with ID starting with 3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb not found: ID does not exist" Nov 04 05:29:35.399331 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.399287 5904 scope.go:122] "RemoveContainer" containerID="7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6" Nov 04 05:29:35.399506 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.399490 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6"} err="failed to get container status \"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6\": rpc error: code = NotFound desc = could not find container \"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6\": container with ID starting with 7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6 not found: ID does not exist" Nov 04 05:29:35.399506 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.399505 5904 scope.go:122] "RemoveContainer" containerID="2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df" Nov 04 05:29:35.399732 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.399717 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df"} err="failed to get container status \"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df\": rpc error: code = NotFound desc = could not find container \"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df\": container with ID starting with 2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df not found: ID does not exist" Nov 04 05:29:35.399732 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.399731 5904 scope.go:122] "RemoveContainer" containerID="33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977" Nov 04 05:29:35.399918 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.399903 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977"} err="failed to get container status \"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977\": rpc error: code = NotFound desc = could not find container \"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977\": container with ID starting with 33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977 not found: ID does not exist" Nov 04 05:29:35.399961 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.399918 5904 scope.go:122] "RemoveContainer" containerID="021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff" Nov 04 05:29:35.400117 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.400102 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff"} err="failed to get container status \"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff\": rpc error: code = NotFound desc = could not find container \"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff\": container with ID starting with 021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff not found: ID does not exist" Nov 04 05:29:35.400164 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.400117 5904 scope.go:122] "RemoveContainer" containerID="522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45" Nov 04 05:29:35.400363 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.400343 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45"} err="failed to get container status \"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45\": rpc error: code = NotFound desc = could not find container \"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45\": container with ID starting with 522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45 not found: ID does not exist" Nov 04 05:29:35.400426 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.400364 5904 scope.go:122] "RemoveContainer" containerID="fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b" Nov 04 05:29:35.400603 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.400587 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b"} err="failed to get container status \"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b\": rpc error: code = NotFound desc = could not find container \"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b\": container with ID starting with fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b not found: ID does not exist" Nov 04 05:29:35.400646 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.400603 5904 scope.go:122] "RemoveContainer" containerID="3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb" Nov 04 05:29:35.400794 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.400779 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb"} err="failed to get container status \"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb\": rpc error: code = NotFound desc = could not find container \"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb\": container with ID starting with 3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb not found: ID does not exist" Nov 04 05:29:35.400836 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.400793 5904 scope.go:122] "RemoveContainer" containerID="7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6" Nov 04 05:29:35.401010 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.400992 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6"} err="failed to get container status \"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6\": rpc error: code = NotFound desc = could not find container \"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6\": container with ID starting with 7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6 not found: ID does not exist" Nov 04 05:29:35.401052 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.401010 5904 scope.go:122] "RemoveContainer" containerID="2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df" Nov 04 05:29:35.401204 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.401190 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df"} err="failed to get container status \"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df\": rpc error: code = NotFound desc = could not find container \"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df\": container with ID starting with 2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df not found: ID does not exist" Nov 04 05:29:35.401283 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.401204 5904 scope.go:122] "RemoveContainer" containerID="33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977" Nov 04 05:29:35.401454 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.401435 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977"} err="failed to get container status \"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977\": rpc error: code = NotFound desc = could not find container \"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977\": container with ID starting with 33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977 not found: ID does not exist" Nov 04 05:29:35.401499 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.401455 5904 scope.go:122] "RemoveContainer" containerID="021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff" Nov 04 05:29:35.401661 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.401641 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff"} err="failed to get container status \"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff\": rpc error: code = NotFound desc = could not find container \"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff\": container with ID starting with 021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff not found: ID does not exist" Nov 04 05:29:35.401661 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.401658 5904 scope.go:122] "RemoveContainer" containerID="522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45" Nov 04 05:29:35.401866 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.401849 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45"} err="failed to get container status \"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45\": rpc error: code = NotFound desc = could not find container \"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45\": container with ID starting with 522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45 not found: ID does not exist" Nov 04 05:29:35.401911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.401866 5904 scope.go:122] "RemoveContainer" containerID="fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b" Nov 04 05:29:35.402072 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.402056 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b"} err="failed to get container status \"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b\": rpc error: code = NotFound desc = could not find container \"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b\": container with ID starting with fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b not found: ID does not exist" Nov 04 05:29:35.402122 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.402071 5904 scope.go:122] "RemoveContainer" containerID="3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb" Nov 04 05:29:35.402319 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.402299 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb"} err="failed to get container status \"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb\": rpc error: code = NotFound desc = could not find container \"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb\": container with ID starting with 3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb not found: ID does not exist" Nov 04 05:29:35.402400 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.402320 5904 scope.go:122] "RemoveContainer" containerID="7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6" Nov 04 05:29:35.402534 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.402520 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6"} err="failed to get container status \"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6\": rpc error: code = NotFound desc = could not find container \"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6\": container with ID starting with 7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6 not found: ID does not exist" Nov 04 05:29:35.402576 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.402535 5904 scope.go:122] "RemoveContainer" containerID="2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df" Nov 04 05:29:35.402725 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.402706 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df"} err="failed to get container status \"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df\": rpc error: code = NotFound desc = could not find container \"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df\": container with ID starting with 2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df not found: ID does not exist" Nov 04 05:29:35.402793 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.402726 5904 scope.go:122] "RemoveContainer" containerID="33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977" Nov 04 05:29:35.402936 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.402920 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977"} err="failed to get container status \"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977\": rpc error: code = NotFound desc = could not find container \"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977\": container with ID starting with 33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977 not found: ID does not exist" Nov 04 05:29:35.402983 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.402937 5904 scope.go:122] "RemoveContainer" containerID="021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff" Nov 04 05:29:35.403153 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.403135 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff"} err="failed to get container status \"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff\": rpc error: code = NotFound desc = could not find container \"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff\": container with ID starting with 021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff not found: ID does not exist" Nov 04 05:29:35.403196 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.403153 5904 scope.go:122] "RemoveContainer" containerID="522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45" Nov 04 05:29:35.403367 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.403351 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45"} err="failed to get container status \"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45\": rpc error: code = NotFound desc = could not find container \"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45\": container with ID starting with 522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45 not found: ID does not exist" Nov 04 05:29:35.403408 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.403368 5904 scope.go:122] "RemoveContainer" containerID="fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b" Nov 04 05:29:35.403561 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.403547 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b"} err="failed to get container status \"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b\": rpc error: code = NotFound desc = could not find container \"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b\": container with ID starting with fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b not found: ID does not exist" Nov 04 05:29:35.403597 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.403563 5904 scope.go:122] "RemoveContainer" containerID="3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb" Nov 04 05:29:35.403743 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.403729 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb"} err="failed to get container status \"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb\": rpc error: code = NotFound desc = could not find container \"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb\": container with ID starting with 3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb not found: ID does not exist" Nov 04 05:29:35.403784 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.403743 5904 scope.go:122] "RemoveContainer" containerID="7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6" Nov 04 05:29:35.403905 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.403890 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6"} err="failed to get container status \"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6\": rpc error: code = NotFound desc = could not find container \"7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6\": container with ID starting with 7849b905f37ca0ae605c51ff08128c7865ec7eba08c837a1ae0544940ea2e6c6 not found: ID does not exist" Nov 04 05:29:35.403945 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.403905 5904 scope.go:122] "RemoveContainer" containerID="2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df" Nov 04 05:29:35.404072 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.404054 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df"} err="failed to get container status \"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df\": rpc error: code = NotFound desc = could not find container \"2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df\": container with ID starting with 2b238bf1be194bad587ccc9a0ed4c89bbda959041be71d6aee8f0ee30da797df not found: ID does not exist" Nov 04 05:29:35.404112 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.404071 5904 scope.go:122] "RemoveContainer" containerID="33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977" Nov 04 05:29:35.404250 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.404215 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977"} err="failed to get container status \"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977\": rpc error: code = NotFound desc = could not find container \"33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977\": container with ID starting with 33f550bd63c073d5bfa18c397db6ce6e773978405c950a009642221cd82d5977 not found: ID does not exist" Nov 04 05:29:35.404298 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.404252 5904 scope.go:122] "RemoveContainer" containerID="021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff" Nov 04 05:29:35.404438 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.404419 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff"} err="failed to get container status \"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff\": rpc error: code = NotFound desc = could not find container \"021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff\": container with ID starting with 021d621a0568c91b47d413d87b71a1a9e0beb283a745bcaa2d740ccc28a3aeff not found: ID does not exist" Nov 04 05:29:35.404511 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.404439 5904 scope.go:122] "RemoveContainer" containerID="522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45" Nov 04 05:29:35.404662 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.404645 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45"} err="failed to get container status \"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45\": rpc error: code = NotFound desc = could not find container \"522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45\": container with ID starting with 522780073befe0fc3a5580a766c4d74d366eee04a91ea9c0e17b94496ee47d45 not found: ID does not exist" Nov 04 05:29:35.404726 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.404672 5904 scope.go:122] "RemoveContainer" containerID="fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b" Nov 04 05:29:35.404876 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.404860 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b"} err="failed to get container status \"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b\": rpc error: code = NotFound desc = could not find container \"fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b\": container with ID starting with fefa68b5de1d4a5d0cb60230093cfbaec0e84a91cb259d95ab9613cb92ce795b not found: ID does not exist" Nov 04 05:29:35.404920 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.404876 5904 scope.go:122] "RemoveContainer" containerID="3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb" Nov 04 05:29:35.405073 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.405054 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb"} err="failed to get container status \"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb\": rpc error: code = NotFound desc = could not find container \"3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb\": container with ID starting with 3b9af4983ce58f53d0ff7d8ce4ff596eb9fbb2ab379314284011fcd74c052bcb not found: ID does not exist" Nov 04 05:29:35.425878 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.425851 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:29:35.425989 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.425977 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.430557 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.430534 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Nov 04 05:29:35.430684 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.430559 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Nov 04 05:29:35.430684 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.430571 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Nov 04 05:29:35.430684 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.430584 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Nov 04 05:29:35.430684 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.430631 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Nov 04 05:29:35.430684 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.430647 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8w4gl\"" Nov 04 05:29:35.430959 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.430711 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Nov 04 05:29:35.430959 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.430722 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Nov 04 05:29:35.430959 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.430748 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Nov 04 05:29:35.430959 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.430921 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Nov 04 05:29:35.430959 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.430928 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Nov 04 05:29:35.430959 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.430943 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Nov 04 05:29:35.431930 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.431913 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-teufh2l7p2i9\"" Nov 04 05:29:35.432020 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.431931 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Nov 04 05:29:35.438568 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.438552 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Nov 04 05:29:35.438672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.438657 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Nov 04 05:29:35.493038 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493008 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493164 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493041 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493164 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493064 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493262 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493161 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493262 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493189 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493262 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493207 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493410 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493291 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493410 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493320 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493410 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493372 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3160ae8d-b254-4663-82e5-689c620f553c-config-out\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493410 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493392 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czvz6\" (UniqueName: \"kubernetes.io/projected/3160ae8d-b254-4663-82e5-689c620f553c-kube-api-access-czvz6\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493519 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493432 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-config\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493519 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493470 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493519 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493495 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493519 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493510 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493642 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493564 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493642 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493614 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-web-config\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493702 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493684 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493742 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493729 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493778 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493767 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3160ae8d-b254-4663-82e5-689c620f553c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.493808 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.493793 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3160ae8d-b254-4663-82e5-689c620f553c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.594542 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.594465 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3160ae8d-b254-4663-82e5-689c620f553c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.594542 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.594501 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3160ae8d-b254-4663-82e5-689c620f553c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.594751 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.594553 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.594751 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.594570 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.594751 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.594595 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.594751 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.594648 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.594751 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.594680 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.594751 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.594707 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.594751 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.594745 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.595121 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.594770 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.595121 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.594823 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3160ae8d-b254-4663-82e5-689c620f553c-config-out\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.595121 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.594848 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czvz6\" (UniqueName: \"kubernetes.io/projected/3160ae8d-b254-4663-82e5-689c620f553c-kube-api-access-czvz6\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.595121 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.594878 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-config\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.595121 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.594935 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.595121 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.594969 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.595121 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.594995 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.595121 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.595014 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3160ae8d-b254-4663-82e5-689c620f553c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.595121 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.595032 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.595121 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.595067 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-web-config\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.595121 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.595126 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.595718 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.595163 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.595718 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.595382 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.595718 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.595460 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.596988 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.596262 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.596988 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.596411 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.596988 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.596836 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.597519 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.597490 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.597863 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.597845 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.598206 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.598187 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.598368 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.598345 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3160ae8d-b254-4663-82e5-689c620f553c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.598810 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.598783 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.598947 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.598924 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.599034 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.598934 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.599410 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.599382 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.599523 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.599500 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-web-config\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.599939 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.599918 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-config\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.600418 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.600400 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3160ae8d-b254-4663-82e5-689c620f553c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.600597 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.600580 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3160ae8d-b254-4663-82e5-689c620f553c-config-out\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.600738 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.600709 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3160ae8d-b254-4663-82e5-689c620f553c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.602536 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.602514 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czvz6\" (UniqueName: \"kubernetes.io/projected/3160ae8d-b254-4663-82e5-689c620f553c-kube-api-access-czvz6\") pod \"prometheus-k8s-0\" (UID: \"3160ae8d-b254-4663-82e5-689c620f553c\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:35.878417 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:35.878341 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:36.191825 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:36.191797 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:29:36.192853 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:29:36.192830 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3160ae8d_b254_4663_82e5_689c620f553c.slice/crio-c3046413aa1a75a2f0d59d55ddf03f17ff2ca455317f972fb5ad266b241f61c0 WatchSource:0}: Error finding container c3046413aa1a75a2f0d59d55ddf03f17ff2ca455317f972fb5ad266b241f61c0: Status 404 returned error can't find the container with id c3046413aa1a75a2f0d59d55ddf03f17ff2ca455317f972fb5ad266b241f61c0 Nov 04 05:29:36.343586 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:36.343558 5904 generic.go:356] "Generic (PLEG): container finished" podID="3160ae8d-b254-4663-82e5-689c620f553c" containerID="d05ef4532b27bf70899fde1376bebc0aaca4953e7acba221a2b756387cf81fe0" exitCode=0 Nov 04 05:29:36.343939 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:36.343647 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3160ae8d-b254-4663-82e5-689c620f553c","Type":"ContainerDied","Data":"d05ef4532b27bf70899fde1376bebc0aaca4953e7acba221a2b756387cf81fe0"} Nov 04 05:29:36.343939 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:36.343719 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3160ae8d-b254-4663-82e5-689c620f553c","Type":"ContainerStarted","Data":"c3046413aa1a75a2f0d59d55ddf03f17ff2ca455317f972fb5ad266b241f61c0"} Nov 04 05:29:37.166341 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:37.166311 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e402f928-dd6c-4f77-acbb-94b57e6b5188" path="/var/lib/kubelet/pods/e402f928-dd6c-4f77-acbb-94b57e6b5188/volumes" Nov 04 05:29:37.351677 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:37.351644 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3160ae8d-b254-4663-82e5-689c620f553c","Type":"ContainerStarted","Data":"29d209fbf202b838615789883ec215fe937a6491e4075e6437a2c09e828f1d57"} Nov 04 05:29:37.352041 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:37.351708 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3160ae8d-b254-4663-82e5-689c620f553c","Type":"ContainerStarted","Data":"f6bacd7a15088e65121f7463126ff17402eda21cae415eafaa2230b798c207fd"} Nov 04 05:29:37.352041 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:37.351737 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3160ae8d-b254-4663-82e5-689c620f553c","Type":"ContainerStarted","Data":"1a7bf115a186c6603def6795925dbc9a6910ffdcde59b8490df3ef52b5090b23"} Nov 04 05:29:37.352041 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:37.351764 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3160ae8d-b254-4663-82e5-689c620f553c","Type":"ContainerStarted","Data":"be3477cbeade20367c6a317e01b94af3b2b2fda11dbe22a22a2b09c741f356e4"} Nov 04 05:29:37.352041 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:37.351794 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3160ae8d-b254-4663-82e5-689c620f553c","Type":"ContainerStarted","Data":"460a9f1a690f52e6df0167587e336398473ad820e67571040fa449e00a9367ee"} Nov 04 05:29:37.352041 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:37.351818 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3160ae8d-b254-4663-82e5-689c620f553c","Type":"ContainerStarted","Data":"48abc9e7f6b525c2b93648006923772e9706a3fd6764e9ad1a8ff073212e6f15"} Nov 04 05:29:37.378550 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:37.378493 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.3581395560000002 podStartE2EDuration="2.378475913s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.336357ms" isStatefulPod=false podCreationTimestamp="2026-11-04 05:29:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:29:37.376973067 +0000 UTC m=+24.334558018" watchObservedRunningTime="2026-11-04 05:29:37.378475913 +0000 UTC m=+24.336060863" Nov 04 05:29:37.504300 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:37.504264 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:37.519662 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:37.519638 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:38.355727 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:38.355699 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:38.370654 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:38.370624 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:38.463472 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:38.463435 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-slrx2" Nov 04 05:29:39.225903 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.225862 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-5f5f49d4fc-6cgqw" podUID="7beaf096-fa6b-4233-a68b-1ab974c2e037" containerName="console" containerID="cri-o://97d36e0f28f1315da3f503de9b66ef3dd55a16de761a7a8fcb120dbe586818fd" gracePeriod=15 Nov 04 05:29:39.232834 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.232799 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6db77f4495-958wf" podUID="e39328c2-e756-4c99-9d13-6fbdccae91cb" containerName="console" containerID="cri-o://38c705efee78d36182ed89aec77d6ceb65252e4482cc100e522c6bfa374ed24e" gracePeriod=15 Nov 04 05:29:39.365182 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.365152 5904 generic.go:356] "Generic (PLEG): container finished" podID="e39328c2-e756-4c99-9d13-6fbdccae91cb" containerID="38c705efee78d36182ed89aec77d6ceb65252e4482cc100e522c6bfa374ed24e" exitCode=0 Nov 04 05:29:39.365626 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.365174 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db77f4495-958wf" event={"ID":"e39328c2-e756-4c99-9d13-6fbdccae91cb","Type":"ContainerDied","Data":"38c705efee78d36182ed89aec77d6ceb65252e4482cc100e522c6bfa374ed24e"} Nov 04 05:29:39.367243 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.367209 5904 generic.go:356] "Generic (PLEG): container finished" podID="7beaf096-fa6b-4233-a68b-1ab974c2e037" containerID="97d36e0f28f1315da3f503de9b66ef3dd55a16de761a7a8fcb120dbe586818fd" exitCode=0 Nov 04 05:29:39.367382 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.367291 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f5f49d4fc-6cgqw" event={"ID":"7beaf096-fa6b-4233-a68b-1ab974c2e037","Type":"ContainerDied","Data":"97d36e0f28f1315da3f503de9b66ef3dd55a16de761a7a8fcb120dbe586818fd"} Nov 04 05:29:39.486190 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.486170 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:39.488918 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.488898 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:39.640874 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.640839 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert\") pod \"7beaf096-fa6b-4233-a68b-1ab974c2e037\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " Nov 04 05:29:39.641030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.640886 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e39328c2-e756-4c99-9d13-6fbdccae91cb-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e39328c2-e756-4c99-9d13-6fbdccae91cb-tmp\") pod \"e39328c2-e756-4c99-9d13-6fbdccae91cb\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " Nov 04 05:29:39.641030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.640925 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca\") pod \"e39328c2-e756-4c99-9d13-6fbdccae91cb\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " Nov 04 05:29:39.641030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.640948 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf\" (UniqueName: \"kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf\") pod \"7beaf096-fa6b-4233-a68b-1ab974c2e037\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " Nov 04 05:29:39.641030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.640977 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config\") pod \"7beaf096-fa6b-4233-a68b-1ab974c2e037\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " Nov 04 05:29:39.641030 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641023 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert\") pod \"7beaf096-fa6b-4233-a68b-1ab974c2e037\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " Nov 04 05:29:39.641335 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641072 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca\") pod \"7beaf096-fa6b-4233-a68b-1ab974c2e037\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " Nov 04 05:29:39.641335 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641118 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle\") pod \"e39328c2-e756-4c99-9d13-6fbdccae91cb\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " Nov 04 05:29:39.641335 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641316 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e39328c2-e756-4c99-9d13-6fbdccae91cb-tmp" pod "e39328c2-e756-4c99-9d13-6fbdccae91cb" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:29:39.641473 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641448 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv\" (UniqueName: \"kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv\") pod \"e39328c2-e756-4c99-9d13-6fbdccae91cb\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " Nov 04 05:29:39.641562 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641450 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config" pod "7beaf096-fa6b-4233-a68b-1ab974c2e037" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:39.641620 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641509 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca" pod "e39328c2-e756-4c99-9d13-6fbdccae91cb" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:39.641620 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641563 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle" pod "e39328c2-e756-4c99-9d13-6fbdccae91cb" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:39.641620 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641601 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert\") pod \"e39328c2-e756-4c99-9d13-6fbdccae91cb\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " Nov 04 05:29:39.641768 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641641 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config\") pod \"7beaf096-fa6b-4233-a68b-1ab974c2e037\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " Nov 04 05:29:39.641768 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641691 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config\") pod \"e39328c2-e756-4c99-9d13-6fbdccae91cb\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " Nov 04 05:29:39.641768 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641726 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca" pod "7beaf096-fa6b-4233-a68b-1ab974c2e037" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:39.641768 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641731 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert\") pod \"e39328c2-e756-4c99-9d13-6fbdccae91cb\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " Nov 04 05:29:39.641972 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641763 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert" pod "7beaf096-fa6b-4233-a68b-1ab974c2e037" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:39.641972 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641799 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle\") pod \"7beaf096-fa6b-4233-a68b-1ab974c2e037\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " Nov 04 05:29:39.641972 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641835 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config\") pod \"e39328c2-e756-4c99-9d13-6fbdccae91cb\" (UID: \"e39328c2-e756-4c99-9d13-6fbdccae91cb\") " Nov 04 05:29:39.641972 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641882 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7beaf096-fa6b-4233-a68b-1ab974c2e037-tmp\" (UniqueName: \"kubernetes.io/empty-dir/7beaf096-fa6b-4233-a68b-1ab974c2e037-tmp\") pod \"7beaf096-fa6b-4233-a68b-1ab974c2e037\" (UID: \"7beaf096-fa6b-4233-a68b-1ab974c2e037\") " Nov 04 05:29:39.642174 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.641969 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert" pod "e39328c2-e756-4c99-9d13-6fbdccae91cb" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:39.642256 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.642167 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7beaf096-fa6b-4233-a68b-1ab974c2e037-tmp" pod "7beaf096-fa6b-4233-a68b-1ab974c2e037" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:29:39.642330 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.642300 5904 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-oauth-serving-cert\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:39.642330 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.642320 5904 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-service-ca\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:39.642434 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.642335 5904 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-trusted-ca-bundle\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:39.642434 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.642349 5904 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-oauth-serving-cert\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:39.642434 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.642362 5904 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7beaf096-fa6b-4233-a68b-1ab974c2e037-tmp\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:39.642434 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.642374 5904 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e39328c2-e756-4c99-9d13-6fbdccae91cb-tmp\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:39.642434 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.642386 5904 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-service-ca\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:39.642434 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.642400 5904 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-config\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:39.642434 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.642420 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config" pod "e39328c2-e756-4c99-9d13-6fbdccae91cb" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:39.642789 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.642511 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle" pod "7beaf096-fa6b-4233-a68b-1ab974c2e037" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:39.643905 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.643880 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert" pod "e39328c2-e756-4c99-9d13-6fbdccae91cb" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:39.644001 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.643906 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert" pod "7beaf096-fa6b-4233-a68b-1ab974c2e037" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:39.644001 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.643927 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv" pod "e39328c2-e756-4c99-9d13-6fbdccae91cb" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb"). InnerVolumeSpecName "kube-api-access-5mwlv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:29:39.644001 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.643915 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf" pod "7beaf096-fa6b-4233-a68b-1ab974c2e037" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037"). InnerVolumeSpecName "kube-api-access-xnhgf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:29:39.644536 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.644517 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config" pod "7beaf096-fa6b-4233-a68b-1ab974c2e037" (UID: "7beaf096-fa6b-4233-a68b-1ab974c2e037"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:39.644614 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.644534 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config" pod "e39328c2-e756-4c99-9d13-6fbdccae91cb" (UID: "e39328c2-e756-4c99-9d13-6fbdccae91cb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:39.743137 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.743063 5904 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-config\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:39.743137 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.743088 5904 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-serving-cert\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:39.743137 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.743098 5904 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7beaf096-fa6b-4233-a68b-1ab974c2e037-trusted-ca-bundle\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:39.743137 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.743107 5904 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e39328c2-e756-4c99-9d13-6fbdccae91cb-console-oauth-config\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:39.743137 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.743117 5904 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-serving-cert\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:39.743137 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.743126 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnhgf\" (UniqueName: \"kubernetes.io/projected/7beaf096-fa6b-4233-a68b-1ab974c2e037-kube-api-access-xnhgf\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:39.743137 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.743135 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mwlv\" (UniqueName: \"kubernetes.io/projected/e39328c2-e756-4c99-9d13-6fbdccae91cb-kube-api-access-5mwlv\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:39.743137 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:39.743143 5904 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7beaf096-fa6b-4233-a68b-1ab974c2e037-console-oauth-config\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:40.372835 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:40.372804 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f5f49d4fc-6cgqw" Nov 04 05:29:40.372835 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:40.372810 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f5f49d4fc-6cgqw" event={"ID":"7beaf096-fa6b-4233-a68b-1ab974c2e037","Type":"ContainerDied","Data":"3f5185ad9371ea13cbae73e1eb02bfa3a76d165dcc331d716aca0b8b2ffdcc12"} Nov 04 05:29:40.373391 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:40.372873 5904 scope.go:122] "RemoveContainer" containerID="97d36e0f28f1315da3f503de9b66ef3dd55a16de761a7a8fcb120dbe586818fd" Nov 04 05:29:40.374298 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:40.374267 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db77f4495-958wf" Nov 04 05:29:40.374370 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:40.374311 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db77f4495-958wf" event={"ID":"e39328c2-e756-4c99-9d13-6fbdccae91cb","Type":"ContainerDied","Data":"88dbe066e70488397c0c8a44b49766afe9dd1661739d3a6f1b29d418b982ea80"} Nov 04 05:29:40.386038 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:40.386019 5904 scope.go:122] "RemoveContainer" containerID="38c705efee78d36182ed89aec77d6ceb65252e4482cc100e522c6bfa374ed24e" Nov 04 05:29:40.398416 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:40.398393 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6db77f4495-958wf"] Nov 04 05:29:40.402080 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:40.402060 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6db77f4495-958wf"] Nov 04 05:29:40.411453 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:40.411432 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f5f49d4fc-6cgqw"] Nov 04 05:29:40.417885 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:40.417863 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f5f49d4fc-6cgqw"] Nov 04 05:29:41.166902 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:41.166866 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7beaf096-fa6b-4233-a68b-1ab974c2e037" path="/var/lib/kubelet/pods/7beaf096-fa6b-4233-a68b-1ab974c2e037/volumes" Nov 04 05:29:41.167390 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:41.167369 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e39328c2-e756-4c99-9d13-6fbdccae91cb" path="/var/lib/kubelet/pods/e39328c2-e756-4c99-9d13-6fbdccae91cb/volumes" Nov 04 05:29:41.550180 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:29:41.550106 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mq5nl" Nov 04 05:29:57.131558 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:29:57.131522 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:30:27.131432 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:30:27.131392 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:30:57.131498 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:30:57.131410 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:31:27.130956 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:31:27.130918 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:31:57.131573 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:31:57.131532 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:32:27.131623 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:32:27.131543 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:32:57.130374 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:32:57.130328 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:33:27.132251 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:33:27.132192 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:33:57.130994 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:33:57.130909 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:34:13.193397 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:34:13.193368 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-slrx2_82660df8-f278-4d7c-b02a-149527e30d27/ovn-acl-logging/0.log" Nov 04 05:34:13.196797 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:34:13.196775 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-slrx2_82660df8-f278-4d7c-b02a-149527e30d27/ovn-acl-logging/0.log" Nov 04 05:34:27.131830 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:34:27.131791 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:34:57.131585 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:34:57.131547 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:35:27.131667 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:35:27.131579 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:35:57.131535 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:35:57.131493 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:36:17.017617 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.017575 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt"] Nov 04 05:36:17.018080 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.017898 5904 cpu_manager.go:414] "containerMap: removing container" podUID="7beaf096-fa6b-4233-a68b-1ab974c2e037" containerName="console" Nov 04 05:36:17.018080 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.017909 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7beaf096-fa6b-4233-a68b-1ab974c2e037" containerName="console" Nov 04 05:36:17.018080 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.017927 5904 cpu_manager.go:414] "containerMap: removing container" podUID="e39328c2-e756-4c99-9d13-6fbdccae91cb" containerName="console" Nov 04 05:36:17.018080 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.017932 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e39328c2-e756-4c99-9d13-6fbdccae91cb" containerName="console" Nov 04 05:36:17.018080 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.017996 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="7beaf096-fa6b-4233-a68b-1ab974c2e037" containerName="console" Nov 04 05:36:17.018080 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.018010 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="e39328c2-e756-4c99-9d13-6fbdccae91cb" containerName="console" Nov 04 05:36:17.021060 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.021042 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" Nov 04 05:36:17.024842 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.024824 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-k82bk\"" Nov 04 05:36:17.027743 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.027720 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Nov 04 05:36:17.027870 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.027842 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Nov 04 05:36:17.029205 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.029184 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt"] Nov 04 05:36:17.141726 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.141695 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7766a580-31a4-4dfb-bd46-7e304375f841-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt\" (UID: \"7766a580-31a4-4dfb-bd46-7e304375f841\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" Nov 04 05:36:17.141905 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.141753 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7766a580-31a4-4dfb-bd46-7e304375f841-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt\" (UID: \"7766a580-31a4-4dfb-bd46-7e304375f841\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" Nov 04 05:36:17.141905 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.141856 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hswng\" (UniqueName: \"kubernetes.io/projected/7766a580-31a4-4dfb-bd46-7e304375f841-kube-api-access-hswng\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt\" (UID: \"7766a580-31a4-4dfb-bd46-7e304375f841\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" Nov 04 05:36:17.242768 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.242728 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7766a580-31a4-4dfb-bd46-7e304375f841-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt\" (UID: \"7766a580-31a4-4dfb-bd46-7e304375f841\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" Nov 04 05:36:17.242963 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.242784 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hswng\" (UniqueName: \"kubernetes.io/projected/7766a580-31a4-4dfb-bd46-7e304375f841-kube-api-access-hswng\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt\" (UID: \"7766a580-31a4-4dfb-bd46-7e304375f841\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" Nov 04 05:36:17.242963 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.242817 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7766a580-31a4-4dfb-bd46-7e304375f841-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt\" (UID: \"7766a580-31a4-4dfb-bd46-7e304375f841\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" Nov 04 05:36:17.243196 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.243172 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7766a580-31a4-4dfb-bd46-7e304375f841-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt\" (UID: \"7766a580-31a4-4dfb-bd46-7e304375f841\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" Nov 04 05:36:17.243260 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.243178 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7766a580-31a4-4dfb-bd46-7e304375f841-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt\" (UID: \"7766a580-31a4-4dfb-bd46-7e304375f841\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" Nov 04 05:36:17.251408 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.251382 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hswng\" (UniqueName: \"kubernetes.io/projected/7766a580-31a4-4dfb-bd46-7e304375f841-kube-api-access-hswng\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt\" (UID: \"7766a580-31a4-4dfb-bd46-7e304375f841\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" Nov 04 05:36:17.337314 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.337265 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" Nov 04 05:36:17.607782 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.607712 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt"] Nov 04 05:36:17.610856 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:36:17.610834 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7766a580_31a4_4dfb_bd46_7e304375f841.slice/crio-0118f6789dc10cc1f2e62297a1823d025589446b586c1ca3450dc89ca3516a9d WatchSource:0}: Error finding container 0118f6789dc10cc1f2e62297a1823d025589446b586c1ca3450dc89ca3516a9d: Status 404 returned error can't find the container with id 0118f6789dc10cc1f2e62297a1823d025589446b586c1ca3450dc89ca3516a9d Nov 04 05:36:17.612890 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.612870 5904 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:36:17.693255 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.693200 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4"] Nov 04 05:36:17.697799 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.697777 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" Nov 04 05:36:17.703985 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.703959 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4"] Nov 04 05:36:17.704145 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.704128 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-jnzjv\"" Nov 04 05:36:17.747031 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.746998 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db2bb7f2-f00e-4565-860e-1a51c1b45f40-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4\" (UID: \"db2bb7f2-f00e-4565-860e-1a51c1b45f40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" Nov 04 05:36:17.747195 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.747055 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pvvw\" (UniqueName: \"kubernetes.io/projected/db2bb7f2-f00e-4565-860e-1a51c1b45f40-kube-api-access-7pvvw\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4\" (UID: \"db2bb7f2-f00e-4565-860e-1a51c1b45f40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" Nov 04 05:36:17.747195 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.747126 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db2bb7f2-f00e-4565-860e-1a51c1b45f40-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4\" (UID: \"db2bb7f2-f00e-4565-860e-1a51c1b45f40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" Nov 04 05:36:17.765966 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.765933 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" event={"ID":"7766a580-31a4-4dfb-bd46-7e304375f841","Type":"ContainerStarted","Data":"0118f6789dc10cc1f2e62297a1823d025589446b586c1ca3450dc89ca3516a9d"} Nov 04 05:36:17.792766 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.792732 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr"] Nov 04 05:36:17.796267 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.796249 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" Nov 04 05:36:17.805203 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.802298 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-jwxc2\"" Nov 04 05:36:17.805203 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.803500 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr"] Nov 04 05:36:17.848335 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.848309 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db2bb7f2-f00e-4565-860e-1a51c1b45f40-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4\" (UID: \"db2bb7f2-f00e-4565-860e-1a51c1b45f40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" Nov 04 05:36:17.848500 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.848357 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pvvw\" (UniqueName: \"kubernetes.io/projected/db2bb7f2-f00e-4565-860e-1a51c1b45f40-kube-api-access-7pvvw\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4\" (UID: \"db2bb7f2-f00e-4565-860e-1a51c1b45f40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" Nov 04 05:36:17.848500 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.848382 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db2bb7f2-f00e-4565-860e-1a51c1b45f40-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4\" (UID: \"db2bb7f2-f00e-4565-860e-1a51c1b45f40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" Nov 04 05:36:17.848500 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.848419 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8gv8\" (UniqueName: \"kubernetes.io/projected/093f37a7-a113-4357-8b08-5966ee43e700-kube-api-access-x8gv8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr\" (UID: \"093f37a7-a113-4357-8b08-5966ee43e700\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" Nov 04 05:36:17.848500 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.848465 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/093f37a7-a113-4357-8b08-5966ee43e700-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr\" (UID: \"093f37a7-a113-4357-8b08-5966ee43e700\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" Nov 04 05:36:17.848730 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.848501 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/093f37a7-a113-4357-8b08-5966ee43e700-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr\" (UID: \"093f37a7-a113-4357-8b08-5966ee43e700\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" Nov 04 05:36:17.848790 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.848752 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db2bb7f2-f00e-4565-860e-1a51c1b45f40-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4\" (UID: \"db2bb7f2-f00e-4565-860e-1a51c1b45f40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" Nov 04 05:36:17.848790 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.848764 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db2bb7f2-f00e-4565-860e-1a51c1b45f40-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4\" (UID: \"db2bb7f2-f00e-4565-860e-1a51c1b45f40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" Nov 04 05:36:17.858262 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.858209 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pvvw\" (UniqueName: \"kubernetes.io/projected/db2bb7f2-f00e-4565-860e-1a51c1b45f40-kube-api-access-7pvvw\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4\" (UID: \"db2bb7f2-f00e-4565-860e-1a51c1b45f40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" Nov 04 05:36:17.891216 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.891191 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v"] Nov 04 05:36:17.894613 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.894596 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" Nov 04 05:36:17.897897 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.897873 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-kx94x\"" Nov 04 05:36:17.902700 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.902679 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v"] Nov 04 05:36:17.949805 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.949776 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29f89a80-ab0d-430c-b6e6-c175f68cb20c-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v\" (UID: \"29f89a80-ab0d-430c-b6e6-c175f68cb20c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" Nov 04 05:36:17.949975 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.949816 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29f89a80-ab0d-430c-b6e6-c175f68cb20c-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v\" (UID: \"29f89a80-ab0d-430c-b6e6-c175f68cb20c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" Nov 04 05:36:17.949975 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.949852 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8gv8\" (UniqueName: \"kubernetes.io/projected/093f37a7-a113-4357-8b08-5966ee43e700-kube-api-access-x8gv8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr\" (UID: \"093f37a7-a113-4357-8b08-5966ee43e700\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" Nov 04 05:36:17.949975 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.949926 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7zlx\" (UniqueName: \"kubernetes.io/projected/29f89a80-ab0d-430c-b6e6-c175f68cb20c-kube-api-access-x7zlx\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v\" (UID: \"29f89a80-ab0d-430c-b6e6-c175f68cb20c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" Nov 04 05:36:17.950142 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.950011 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/093f37a7-a113-4357-8b08-5966ee43e700-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr\" (UID: \"093f37a7-a113-4357-8b08-5966ee43e700\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" Nov 04 05:36:17.950142 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.950052 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/093f37a7-a113-4357-8b08-5966ee43e700-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr\" (UID: \"093f37a7-a113-4357-8b08-5966ee43e700\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" Nov 04 05:36:17.950368 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.950350 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/093f37a7-a113-4357-8b08-5966ee43e700-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr\" (UID: \"093f37a7-a113-4357-8b08-5966ee43e700\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" Nov 04 05:36:17.950439 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.950395 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/093f37a7-a113-4357-8b08-5966ee43e700-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr\" (UID: \"093f37a7-a113-4357-8b08-5966ee43e700\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" Nov 04 05:36:17.958199 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:17.958170 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8gv8\" (UniqueName: \"kubernetes.io/projected/093f37a7-a113-4357-8b08-5966ee43e700-kube-api-access-x8gv8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr\" (UID: \"093f37a7-a113-4357-8b08-5966ee43e700\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" Nov 04 05:36:18.016159 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:18.016130 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" Nov 04 05:36:18.050974 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:18.050934 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29f89a80-ab0d-430c-b6e6-c175f68cb20c-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v\" (UID: \"29f89a80-ab0d-430c-b6e6-c175f68cb20c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" Nov 04 05:36:18.051416 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:18.051000 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29f89a80-ab0d-430c-b6e6-c175f68cb20c-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v\" (UID: \"29f89a80-ab0d-430c-b6e6-c175f68cb20c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" Nov 04 05:36:18.051416 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:18.051038 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7zlx\" (UniqueName: \"kubernetes.io/projected/29f89a80-ab0d-430c-b6e6-c175f68cb20c-kube-api-access-x7zlx\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v\" (UID: \"29f89a80-ab0d-430c-b6e6-c175f68cb20c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" Nov 04 05:36:18.051416 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:18.051307 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29f89a80-ab0d-430c-b6e6-c175f68cb20c-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v\" (UID: \"29f89a80-ab0d-430c-b6e6-c175f68cb20c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" Nov 04 05:36:18.051644 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:18.051546 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29f89a80-ab0d-430c-b6e6-c175f68cb20c-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v\" (UID: \"29f89a80-ab0d-430c-b6e6-c175f68cb20c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" Nov 04 05:36:18.059812 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:18.059787 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7zlx\" (UniqueName: \"kubernetes.io/projected/29f89a80-ab0d-430c-b6e6-c175f68cb20c-kube-api-access-x7zlx\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v\" (UID: \"29f89a80-ab0d-430c-b6e6-c175f68cb20c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" Nov 04 05:36:18.114153 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:18.114082 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" Nov 04 05:36:18.212155 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:18.212123 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" Nov 04 05:36:18.296114 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:18.296086 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4"] Nov 04 05:36:18.300961 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:36:18.300911 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb2bb7f2_f00e_4565_860e_1a51c1b45f40.slice/crio-e80e790949b4b9635903e1f227bff376553646aa725a2f401194f1ef52ffeebd WatchSource:0}: Error finding container e80e790949b4b9635903e1f227bff376553646aa725a2f401194f1ef52ffeebd: Status 404 returned error can't find the container with id e80e790949b4b9635903e1f227bff376553646aa725a2f401194f1ef52ffeebd Nov 04 05:36:18.393896 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:18.393824 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr"] Nov 04 05:36:18.397480 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:36:18.397441 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod093f37a7_a113_4357_8b08_5966ee43e700.slice/crio-2b63250e15974c63cb3a20dec04d2d3c9f7e6aee6061607b86cd35e343af8979 WatchSource:0}: Error finding container 2b63250e15974c63cb3a20dec04d2d3c9f7e6aee6061607b86cd35e343af8979: Status 404 returned error can't find the container with id 2b63250e15974c63cb3a20dec04d2d3c9f7e6aee6061607b86cd35e343af8979 Nov 04 05:36:18.486846 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:18.486817 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v"] Nov 04 05:36:18.771597 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:18.771487 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" event={"ID":"093f37a7-a113-4357-8b08-5966ee43e700","Type":"ContainerStarted","Data":"2b63250e15974c63cb3a20dec04d2d3c9f7e6aee6061607b86cd35e343af8979"} Nov 04 05:36:18.772846 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:18.772811 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" event={"ID":"db2bb7f2-f00e-4565-860e-1a51c1b45f40","Type":"ContainerStarted","Data":"e80e790949b4b9635903e1f227bff376553646aa725a2f401194f1ef52ffeebd"} Nov 04 05:36:18.773946 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:18.773920 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" event={"ID":"29f89a80-ab0d-430c-b6e6-c175f68cb20c","Type":"ContainerStarted","Data":"410c712159363d2c4948eb2675f74352751e8153ed417f21dae646fde2492f70"} Nov 04 05:36:25.816817 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:25.816777 5904 generic.go:356] "Generic (PLEG): container finished" podID="7766a580-31a4-4dfb-bd46-7e304375f841" containerID="62bc5633b6389301d5cb71fca57805d2a1d2a4454a64df77b05a8910c7cb6484" exitCode=0 Nov 04 05:36:25.817323 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:25.816863 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" event={"ID":"7766a580-31a4-4dfb-bd46-7e304375f841","Type":"ContainerDied","Data":"62bc5633b6389301d5cb71fca57805d2a1d2a4454a64df77b05a8910c7cb6484"} Nov 04 05:36:27.137170 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:36:27.137116 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:36:27.827796 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:27.827758 5904 generic.go:356] "Generic (PLEG): container finished" podID="093f37a7-a113-4357-8b08-5966ee43e700" containerID="7245521abfbb50c77c7bb5ed2307e241d8d60de9c99604d72b5d5183810fe781" exitCode=0 Nov 04 05:36:27.827991 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:27.827850 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" event={"ID":"093f37a7-a113-4357-8b08-5966ee43e700","Type":"ContainerDied","Data":"7245521abfbb50c77c7bb5ed2307e241d8d60de9c99604d72b5d5183810fe781"} Nov 04 05:36:27.829724 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:27.829695 5904 generic.go:356] "Generic (PLEG): container finished" podID="db2bb7f2-f00e-4565-860e-1a51c1b45f40" containerID="03ef7c5c5040e344f7b06571969f2db6ba94f5f083395fd2d2bb864f572bfe44" exitCode=0 Nov 04 05:36:27.829843 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:27.829793 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" event={"ID":"db2bb7f2-f00e-4565-860e-1a51c1b45f40","Type":"ContainerDied","Data":"03ef7c5c5040e344f7b06571969f2db6ba94f5f083395fd2d2bb864f572bfe44"} Nov 04 05:36:27.831869 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:27.831846 5904 generic.go:356] "Generic (PLEG): container finished" podID="29f89a80-ab0d-430c-b6e6-c175f68cb20c" containerID="688f4547dfaa05260900c57221490101a5565ba53ddbe00f4e1e548a10f8a296" exitCode=0 Nov 04 05:36:27.831966 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:27.831888 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" event={"ID":"29f89a80-ab0d-430c-b6e6-c175f68cb20c","Type":"ContainerDied","Data":"688f4547dfaa05260900c57221490101a5565ba53ddbe00f4e1e548a10f8a296"} Nov 04 05:36:28.839986 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:28.839956 5904 generic.go:356] "Generic (PLEG): container finished" podID="7766a580-31a4-4dfb-bd46-7e304375f841" containerID="affd6aacbcd7e164a968033693a3bbc25307230be92499651c2d0c86f7295293" exitCode=0 Nov 04 05:36:28.840359 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:28.840040 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" event={"ID":"7766a580-31a4-4dfb-bd46-7e304375f841","Type":"ContainerDied","Data":"affd6aacbcd7e164a968033693a3bbc25307230be92499651c2d0c86f7295293"} Nov 04 05:36:28.842000 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:28.841956 5904 generic.go:356] "Generic (PLEG): container finished" podID="093f37a7-a113-4357-8b08-5966ee43e700" containerID="93f3d471c84266da6a8f8de028d4bbe9a056c4ddc7e9d1906f4cd935255e6ce5" exitCode=0 Nov 04 05:36:28.842087 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:28.842068 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" event={"ID":"093f37a7-a113-4357-8b08-5966ee43e700","Type":"ContainerDied","Data":"93f3d471c84266da6a8f8de028d4bbe9a056c4ddc7e9d1906f4cd935255e6ce5"} Nov 04 05:36:29.849969 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:29.849931 5904 generic.go:356] "Generic (PLEG): container finished" podID="db2bb7f2-f00e-4565-860e-1a51c1b45f40" containerID="d49b5e5af7adf34b9506dc0cadc4a3127776104b0af5fc57e040a5720d861b15" exitCode=0 Nov 04 05:36:29.850457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:29.850028 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" event={"ID":"db2bb7f2-f00e-4565-860e-1a51c1b45f40","Type":"ContainerDied","Data":"d49b5e5af7adf34b9506dc0cadc4a3127776104b0af5fc57e040a5720d861b15"} Nov 04 05:36:29.852517 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:29.852491 5904 generic.go:356] "Generic (PLEG): container finished" podID="29f89a80-ab0d-430c-b6e6-c175f68cb20c" containerID="1f9b60a62a978d7173630c571b0e24d630d8e6fb08225ccd6c8084d37dd45b25" exitCode=0 Nov 04 05:36:29.852642 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:29.852548 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" event={"ID":"29f89a80-ab0d-430c-b6e6-c175f68cb20c","Type":"ContainerDied","Data":"1f9b60a62a978d7173630c571b0e24d630d8e6fb08225ccd6c8084d37dd45b25"} Nov 04 05:36:38.899943 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:38.899904 5904 generic.go:356] "Generic (PLEG): container finished" podID="7766a580-31a4-4dfb-bd46-7e304375f841" containerID="8fd0f48ef8beb3b7c4218df296b54512ca49cb0bf1d9486f2e447fee5af1fb4f" exitCode=0 Nov 04 05:36:38.900503 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:38.899996 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" event={"ID":"7766a580-31a4-4dfb-bd46-7e304375f841","Type":"ContainerDied","Data":"8fd0f48ef8beb3b7c4218df296b54512ca49cb0bf1d9486f2e447fee5af1fb4f"} Nov 04 05:36:38.902441 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:38.902411 5904 generic.go:356] "Generic (PLEG): container finished" podID="093f37a7-a113-4357-8b08-5966ee43e700" containerID="b56ff06840cb8df9124846374868b44e1e6d073cfdc23784c1919c8661942f42" exitCode=0 Nov 04 05:36:38.902560 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:38.902495 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" event={"ID":"093f37a7-a113-4357-8b08-5966ee43e700","Type":"ContainerDied","Data":"b56ff06840cb8df9124846374868b44e1e6d073cfdc23784c1919c8661942f42"} Nov 04 05:36:39.908600 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:39.908565 5904 generic.go:356] "Generic (PLEG): container finished" podID="db2bb7f2-f00e-4565-860e-1a51c1b45f40" containerID="5fadbdedcd3efe27623111566b4d56ce6a312e6617713fc2ee73627195f3c0ef" exitCode=0 Nov 04 05:36:39.909033 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:39.908654 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" event={"ID":"db2bb7f2-f00e-4565-860e-1a51c1b45f40","Type":"ContainerDied","Data":"5fadbdedcd3efe27623111566b4d56ce6a312e6617713fc2ee73627195f3c0ef"} Nov 04 05:36:39.910678 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:39.910656 5904 generic.go:356] "Generic (PLEG): container finished" podID="29f89a80-ab0d-430c-b6e6-c175f68cb20c" containerID="2f88efb7fad031d6c5b78fcddec200bfc7b678b012364aab63c1e833fae5c915" exitCode=0 Nov 04 05:36:39.910798 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:39.910739 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" event={"ID":"29f89a80-ab0d-430c-b6e6-c175f68cb20c","Type":"ContainerDied","Data":"2f88efb7fad031d6c5b78fcddec200bfc7b678b012364aab63c1e833fae5c915"} Nov 04 05:36:40.037095 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.037071 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" Nov 04 05:36:40.055512 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.055480 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" Nov 04 05:36:40.175814 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.175723 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7766a580-31a4-4dfb-bd46-7e304375f841-bundle\" (UniqueName: \"kubernetes.io/empty-dir/7766a580-31a4-4dfb-bd46-7e304375f841-bundle\") pod \"7766a580-31a4-4dfb-bd46-7e304375f841\" (UID: \"7766a580-31a4-4dfb-bd46-7e304375f841\") " Nov 04 05:36:40.175814 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.175771 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7766a580-31a4-4dfb-bd46-7e304375f841-kube-api-access-hswng\" (UniqueName: \"kubernetes.io/projected/7766a580-31a4-4dfb-bd46-7e304375f841-kube-api-access-hswng\") pod \"7766a580-31a4-4dfb-bd46-7e304375f841\" (UID: \"7766a580-31a4-4dfb-bd46-7e304375f841\") " Nov 04 05:36:40.175814 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.175791 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7766a580-31a4-4dfb-bd46-7e304375f841-util\" (UniqueName: \"kubernetes.io/empty-dir/7766a580-31a4-4dfb-bd46-7e304375f841-util\") pod \"7766a580-31a4-4dfb-bd46-7e304375f841\" (UID: \"7766a580-31a4-4dfb-bd46-7e304375f841\") " Nov 04 05:36:40.176099 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.175826 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/093f37a7-a113-4357-8b08-5966ee43e700-bundle\" (UniqueName: \"kubernetes.io/empty-dir/093f37a7-a113-4357-8b08-5966ee43e700-bundle\") pod \"093f37a7-a113-4357-8b08-5966ee43e700\" (UID: \"093f37a7-a113-4357-8b08-5966ee43e700\") " Nov 04 05:36:40.176099 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.175879 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/093f37a7-a113-4357-8b08-5966ee43e700-kube-api-access-x8gv8\" (UniqueName: \"kubernetes.io/projected/093f37a7-a113-4357-8b08-5966ee43e700-kube-api-access-x8gv8\") pod \"093f37a7-a113-4357-8b08-5966ee43e700\" (UID: \"093f37a7-a113-4357-8b08-5966ee43e700\") " Nov 04 05:36:40.176099 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.175920 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/093f37a7-a113-4357-8b08-5966ee43e700-util\" (UniqueName: \"kubernetes.io/empty-dir/093f37a7-a113-4357-8b08-5966ee43e700-util\") pod \"093f37a7-a113-4357-8b08-5966ee43e700\" (UID: \"093f37a7-a113-4357-8b08-5966ee43e700\") " Nov 04 05:36:40.176338 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.176221 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7766a580-31a4-4dfb-bd46-7e304375f841-bundle" pod "7766a580-31a4-4dfb-bd46-7e304375f841" (UID: "7766a580-31a4-4dfb-bd46-7e304375f841"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:36:40.176549 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.176525 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/093f37a7-a113-4357-8b08-5966ee43e700-bundle" pod "093f37a7-a113-4357-8b08-5966ee43e700" (UID: "093f37a7-a113-4357-8b08-5966ee43e700"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:36:40.178150 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.178113 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/093f37a7-a113-4357-8b08-5966ee43e700-kube-api-access-x8gv8" pod "093f37a7-a113-4357-8b08-5966ee43e700" (UID: "093f37a7-a113-4357-8b08-5966ee43e700"). InnerVolumeSpecName "kube-api-access-x8gv8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:36:40.178454 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.178439 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7766a580-31a4-4dfb-bd46-7e304375f841-kube-api-access-hswng" pod "7766a580-31a4-4dfb-bd46-7e304375f841" (UID: "7766a580-31a4-4dfb-bd46-7e304375f841"). InnerVolumeSpecName "kube-api-access-hswng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:36:40.180910 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.180886 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7766a580-31a4-4dfb-bd46-7e304375f841-util" pod "7766a580-31a4-4dfb-bd46-7e304375f841" (UID: "7766a580-31a4-4dfb-bd46-7e304375f841"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:36:40.181551 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.181525 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/093f37a7-a113-4357-8b08-5966ee43e700-util" pod "093f37a7-a113-4357-8b08-5966ee43e700" (UID: "093f37a7-a113-4357-8b08-5966ee43e700"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:36:40.277102 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.277065 5904 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/093f37a7-a113-4357-8b08-5966ee43e700-util\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:36:40.277102 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.277097 5904 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7766a580-31a4-4dfb-bd46-7e304375f841-bundle\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:36:40.277102 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.277108 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hswng\" (UniqueName: \"kubernetes.io/projected/7766a580-31a4-4dfb-bd46-7e304375f841-kube-api-access-hswng\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:36:40.277350 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.277118 5904 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7766a580-31a4-4dfb-bd46-7e304375f841-util\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:36:40.277350 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.277127 5904 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/093f37a7-a113-4357-8b08-5966ee43e700-bundle\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:36:40.277350 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.277135 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8gv8\" (UniqueName: \"kubernetes.io/projected/093f37a7-a113-4357-8b08-5966ee43e700-kube-api-access-x8gv8\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:36:40.916105 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.916076 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" Nov 04 05:36:40.916529 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.916076 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644sznbt" event={"ID":"7766a580-31a4-4dfb-bd46-7e304375f841","Type":"ContainerDied","Data":"0118f6789dc10cc1f2e62297a1823d025589446b586c1ca3450dc89ca3516a9d"} Nov 04 05:36:40.916529 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.916210 5904 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0118f6789dc10cc1f2e62297a1823d025589446b586c1ca3450dc89ca3516a9d" Nov 04 05:36:40.918004 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.917985 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" Nov 04 05:36:40.918148 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.918005 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxrnnr" event={"ID":"093f37a7-a113-4357-8b08-5966ee43e700","Type":"ContainerDied","Data":"2b63250e15974c63cb3a20dec04d2d3c9f7e6aee6061607b86cd35e343af8979"} Nov 04 05:36:40.918148 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:40.918050 5904 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b63250e15974c63cb3a20dec04d2d3c9f7e6aee6061607b86cd35e343af8979" Nov 04 05:36:41.058640 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.058620 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" Nov 04 05:36:41.060623 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.060606 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" Nov 04 05:36:41.185573 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.185500 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db2bb7f2-f00e-4565-860e-1a51c1b45f40-kube-api-access-7pvvw\" (UniqueName: \"kubernetes.io/projected/db2bb7f2-f00e-4565-860e-1a51c1b45f40-kube-api-access-7pvvw\") pod \"db2bb7f2-f00e-4565-860e-1a51c1b45f40\" (UID: \"db2bb7f2-f00e-4565-860e-1a51c1b45f40\") " Nov 04 05:36:41.185573 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.185567 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29f89a80-ab0d-430c-b6e6-c175f68cb20c-bundle\" (UniqueName: \"kubernetes.io/empty-dir/29f89a80-ab0d-430c-b6e6-c175f68cb20c-bundle\") pod \"29f89a80-ab0d-430c-b6e6-c175f68cb20c\" (UID: \"29f89a80-ab0d-430c-b6e6-c175f68cb20c\") " Nov 04 05:36:41.185797 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.185680 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db2bb7f2-f00e-4565-860e-1a51c1b45f40-util\" (UniqueName: \"kubernetes.io/empty-dir/db2bb7f2-f00e-4565-860e-1a51c1b45f40-util\") pod \"db2bb7f2-f00e-4565-860e-1a51c1b45f40\" (UID: \"db2bb7f2-f00e-4565-860e-1a51c1b45f40\") " Nov 04 05:36:41.185797 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.185732 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db2bb7f2-f00e-4565-860e-1a51c1b45f40-bundle\" (UniqueName: \"kubernetes.io/empty-dir/db2bb7f2-f00e-4565-860e-1a51c1b45f40-bundle\") pod \"db2bb7f2-f00e-4565-860e-1a51c1b45f40\" (UID: \"db2bb7f2-f00e-4565-860e-1a51c1b45f40\") " Nov 04 05:36:41.185915 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.185797 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29f89a80-ab0d-430c-b6e6-c175f68cb20c-kube-api-access-x7zlx\" (UniqueName: \"kubernetes.io/projected/29f89a80-ab0d-430c-b6e6-c175f68cb20c-kube-api-access-x7zlx\") pod \"29f89a80-ab0d-430c-b6e6-c175f68cb20c\" (UID: \"29f89a80-ab0d-430c-b6e6-c175f68cb20c\") " Nov 04 05:36:41.185915 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.185835 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29f89a80-ab0d-430c-b6e6-c175f68cb20c-util\" (UniqueName: \"kubernetes.io/empty-dir/29f89a80-ab0d-430c-b6e6-c175f68cb20c-util\") pod \"29f89a80-ab0d-430c-b6e6-c175f68cb20c\" (UID: \"29f89a80-ab0d-430c-b6e6-c175f68cb20c\") " Nov 04 05:36:41.186261 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.186205 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29f89a80-ab0d-430c-b6e6-c175f68cb20c-bundle" pod "29f89a80-ab0d-430c-b6e6-c175f68cb20c" (UID: "29f89a80-ab0d-430c-b6e6-c175f68cb20c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:36:41.186389 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.186362 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db2bb7f2-f00e-4565-860e-1a51c1b45f40-bundle" pod "db2bb7f2-f00e-4565-860e-1a51c1b45f40" (UID: "db2bb7f2-f00e-4565-860e-1a51c1b45f40"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:36:41.187789 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.187753 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db2bb7f2-f00e-4565-860e-1a51c1b45f40-kube-api-access-7pvvw" pod "db2bb7f2-f00e-4565-860e-1a51c1b45f40" (UID: "db2bb7f2-f00e-4565-860e-1a51c1b45f40"). InnerVolumeSpecName "kube-api-access-7pvvw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:36:41.187902 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.187836 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29f89a80-ab0d-430c-b6e6-c175f68cb20c-kube-api-access-x7zlx" pod "29f89a80-ab0d-430c-b6e6-c175f68cb20c" (UID: "29f89a80-ab0d-430c-b6e6-c175f68cb20c"). InnerVolumeSpecName "kube-api-access-x7zlx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:36:41.190541 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.190514 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29f89a80-ab0d-430c-b6e6-c175f68cb20c-util" pod "29f89a80-ab0d-430c-b6e6-c175f68cb20c" (UID: "29f89a80-ab0d-430c-b6e6-c175f68cb20c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:36:41.194059 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.193944 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db2bb7f2-f00e-4565-860e-1a51c1b45f40-util" pod "db2bb7f2-f00e-4565-860e-1a51c1b45f40" (UID: "db2bb7f2-f00e-4565-860e-1a51c1b45f40"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:36:41.286953 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.286914 5904 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db2bb7f2-f00e-4565-860e-1a51c1b45f40-bundle\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:36:41.286953 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.286947 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7zlx\" (UniqueName: \"kubernetes.io/projected/29f89a80-ab0d-430c-b6e6-c175f68cb20c-kube-api-access-x7zlx\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:36:41.286953 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.286958 5904 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29f89a80-ab0d-430c-b6e6-c175f68cb20c-util\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:36:41.287172 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.286967 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pvvw\" (UniqueName: \"kubernetes.io/projected/db2bb7f2-f00e-4565-860e-1a51c1b45f40-kube-api-access-7pvvw\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:36:41.287172 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.286975 5904 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29f89a80-ab0d-430c-b6e6-c175f68cb20c-bundle\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:36:41.287172 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.286984 5904 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db2bb7f2-f00e-4565-860e-1a51c1b45f40-util\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:36:41.923122 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.923091 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" event={"ID":"db2bb7f2-f00e-4565-860e-1a51c1b45f40","Type":"ContainerDied","Data":"e80e790949b4b9635903e1f227bff376553646aa725a2f401194f1ef52ffeebd"} Nov 04 05:36:41.923572 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.923122 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s6dt4" Nov 04 05:36:41.923572 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.923140 5904 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e80e790949b4b9635903e1f227bff376553646aa725a2f401194f1ef52ffeebd" Nov 04 05:36:41.925049 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.925026 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" event={"ID":"29f89a80-ab0d-430c-b6e6-c175f68cb20c","Type":"ContainerDied","Data":"410c712159363d2c4948eb2675f74352751e8153ed417f21dae646fde2492f70"} Nov 04 05:36:41.925167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.925059 5904 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="410c712159363d2c4948eb2675f74352751e8153ed417f21dae646fde2492f70" Nov 04 05:36:41.925167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:41.925067 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c677p2v" Nov 04 05:36:43.609235 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609190 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8"] Nov 04 05:36:43.609673 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609628 5904 cpu_manager.go:414] "containerMap: removing container" podUID="7766a580-31a4-4dfb-bd46-7e304375f841" containerName="extract" Nov 04 05:36:43.609673 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609643 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7766a580-31a4-4dfb-bd46-7e304375f841" containerName="extract" Nov 04 05:36:43.609673 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609655 5904 cpu_manager.go:414] "containerMap: removing container" podUID="093f37a7-a113-4357-8b08-5966ee43e700" containerName="pull" Nov 04 05:36:43.609673 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609660 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="093f37a7-a113-4357-8b08-5966ee43e700" containerName="pull" Nov 04 05:36:43.609673 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609670 5904 cpu_manager.go:414] "containerMap: removing container" podUID="29f89a80-ab0d-430c-b6e6-c175f68cb20c" containerName="pull" Nov 04 05:36:43.609673 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609675 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29f89a80-ab0d-430c-b6e6-c175f68cb20c" containerName="pull" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609681 5904 cpu_manager.go:414] "containerMap: removing container" podUID="db2bb7f2-f00e-4565-860e-1a51c1b45f40" containerName="util" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609686 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db2bb7f2-f00e-4565-860e-1a51c1b45f40" containerName="util" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609697 5904 cpu_manager.go:414] "containerMap: removing container" podUID="7766a580-31a4-4dfb-bd46-7e304375f841" containerName="util" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609702 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7766a580-31a4-4dfb-bd46-7e304375f841" containerName="util" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609708 5904 cpu_manager.go:414] "containerMap: removing container" podUID="db2bb7f2-f00e-4565-860e-1a51c1b45f40" containerName="pull" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609714 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db2bb7f2-f00e-4565-860e-1a51c1b45f40" containerName="pull" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609720 5904 cpu_manager.go:414] "containerMap: removing container" podUID="29f89a80-ab0d-430c-b6e6-c175f68cb20c" containerName="extract" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609724 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29f89a80-ab0d-430c-b6e6-c175f68cb20c" containerName="extract" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609733 5904 cpu_manager.go:414] "containerMap: removing container" podUID="093f37a7-a113-4357-8b08-5966ee43e700" containerName="util" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609737 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="093f37a7-a113-4357-8b08-5966ee43e700" containerName="util" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609743 5904 cpu_manager.go:414] "containerMap: removing container" podUID="29f89a80-ab0d-430c-b6e6-c175f68cb20c" containerName="util" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609748 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29f89a80-ab0d-430c-b6e6-c175f68cb20c" containerName="util" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609754 5904 cpu_manager.go:414] "containerMap: removing container" podUID="093f37a7-a113-4357-8b08-5966ee43e700" containerName="extract" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609758 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="093f37a7-a113-4357-8b08-5966ee43e700" containerName="extract" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609767 5904 cpu_manager.go:414] "containerMap: removing container" podUID="db2bb7f2-f00e-4565-860e-1a51c1b45f40" containerName="extract" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609771 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db2bb7f2-f00e-4565-860e-1a51c1b45f40" containerName="extract" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609778 5904 cpu_manager.go:414] "containerMap: removing container" podUID="7766a580-31a4-4dfb-bd46-7e304375f841" containerName="pull" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609782 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7766a580-31a4-4dfb-bd46-7e304375f841" containerName="pull" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609827 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="29f89a80-ab0d-430c-b6e6-c175f68cb20c" containerName="extract" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609836 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="093f37a7-a113-4357-8b08-5966ee43e700" containerName="extract" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609843 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="db2bb7f2-f00e-4565-860e-1a51c1b45f40" containerName="extract" Nov 04 05:36:43.610007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.609851 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="7766a580-31a4-4dfb-bd46-7e304375f841" containerName="extract" Nov 04 05:36:43.615142 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.615124 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.643775 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.643737 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8"] Nov 04 05:36:43.692598 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.692556 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\"" Nov 04 05:36:43.692598 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.692562 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" Nov 04 05:36:43.692817 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.692695 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-vkh6g\"" Nov 04 05:36:43.692817 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.692706 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" Nov 04 05:36:43.692817 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.692716 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" Nov 04 05:36:43.692817 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.692773 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" Nov 04 05:36:43.710090 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.710063 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-tmp\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.710155 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.710100 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-webhook-cert\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.710210 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.710197 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.710283 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.710267 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-metrics-cert\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.710318 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.710305 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-manifests\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.710367 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.710350 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7npp\" (UniqueName: \"kubernetes.io/projected/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-kube-api-access-r7npp\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.810899 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.810865 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7npp\" (UniqueName: \"kubernetes.io/projected/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-kube-api-access-r7npp\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.811057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.810914 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-tmp\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.811057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.810936 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-webhook-cert\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.811057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.810972 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.811057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.810987 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-metrics-cert\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.811057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.811010 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-manifests\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.811481 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.811453 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-manifests\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.813334 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.813309 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-tmp\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.813644 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.813620 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-metrics-cert\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.813764 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.813746 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-webhook-cert\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.813818 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.813803 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.819929 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.819902 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7npp\" (UniqueName: \"kubernetes.io/projected/ac9c1355-171c-413a-9e9a-6c6d3dc92da1-kube-api-access-r7npp\") pod \"opendatahub-operator-controller-manager-56cd9ffd8-ph9j8\" (UID: \"ac9c1355-171c-413a-9e9a-6c6d3dc92da1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:43.932264 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:43.932158 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:44.307557 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:44.307385 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8"] Nov 04 05:36:44.310035 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:36:44.310000 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac9c1355_171c_413a_9e9a_6c6d3dc92da1.slice/crio-668268d7779fd888089e5740fad435c1975e03270d63c7cef6e6f8262564d8f7 WatchSource:0}: Error finding container 668268d7779fd888089e5740fad435c1975e03270d63c7cef6e6f8262564d8f7: Status 404 returned error can't find the container with id 668268d7779fd888089e5740fad435c1975e03270d63c7cef6e6f8262564d8f7 Nov 04 05:36:44.414197 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:44.414162 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-k2r4j"] Nov 04 05:36:44.446178 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:44.446149 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-k2r4j"] Nov 04 05:36:44.446352 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:44.446253 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k2r4j" Nov 04 05:36:44.451374 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:44.451355 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Nov 04 05:36:44.451464 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:44.451448 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Nov 04 05:36:44.519108 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:44.519077 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-92tgm\"" Nov 04 05:36:44.617976 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:44.617936 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kftbj\" (UniqueName: \"kubernetes.io/projected/323e147c-bf76-4cfb-9fd0-d89059dbb309-kube-api-access-kftbj\") pod \"cert-manager-webhook-5dbbbf75cc-k2r4j\" (UID: \"323e147c-bf76-4cfb-9fd0-d89059dbb309\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k2r4j" Nov 04 05:36:44.618392 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:44.618009 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/323e147c-bf76-4cfb-9fd0-d89059dbb309-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-k2r4j\" (UID: \"323e147c-bf76-4cfb-9fd0-d89059dbb309\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k2r4j" Nov 04 05:36:44.719367 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:44.719328 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kftbj\" (UniqueName: \"kubernetes.io/projected/323e147c-bf76-4cfb-9fd0-d89059dbb309-kube-api-access-kftbj\") pod \"cert-manager-webhook-5dbbbf75cc-k2r4j\" (UID: \"323e147c-bf76-4cfb-9fd0-d89059dbb309\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k2r4j" Nov 04 05:36:44.719549 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:44.719471 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/323e147c-bf76-4cfb-9fd0-d89059dbb309-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-k2r4j\" (UID: \"323e147c-bf76-4cfb-9fd0-d89059dbb309\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k2r4j" Nov 04 05:36:44.727172 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:44.727142 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/323e147c-bf76-4cfb-9fd0-d89059dbb309-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-k2r4j\" (UID: \"323e147c-bf76-4cfb-9fd0-d89059dbb309\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k2r4j" Nov 04 05:36:44.727687 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:44.727665 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kftbj\" (UniqueName: \"kubernetes.io/projected/323e147c-bf76-4cfb-9fd0-d89059dbb309-kube-api-access-kftbj\") pod \"cert-manager-webhook-5dbbbf75cc-k2r4j\" (UID: \"323e147c-bf76-4cfb-9fd0-d89059dbb309\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k2r4j" Nov 04 05:36:44.786600 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:44.786574 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k2r4j" Nov 04 05:36:44.944903 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:44.944827 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" event={"ID":"ac9c1355-171c-413a-9e9a-6c6d3dc92da1","Type":"ContainerStarted","Data":"668268d7779fd888089e5740fad435c1975e03270d63c7cef6e6f8262564d8f7"} Nov 04 05:36:45.065869 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:45.065820 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-k2r4j"] Nov 04 05:36:45.078517 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:36:45.078480 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod323e147c_bf76_4cfb_9fd0_d89059dbb309.slice/crio-0257cc713afafa05f300097e4560c9a945462cd60fd7c72876a8516c89c8ed09 WatchSource:0}: Error finding container 0257cc713afafa05f300097e4560c9a945462cd60fd7c72876a8516c89c8ed09: Status 404 returned error can't find the container with id 0257cc713afafa05f300097e4560c9a945462cd60fd7c72876a8516c89c8ed09 Nov 04 05:36:45.951750 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:45.951717 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k2r4j" event={"ID":"323e147c-bf76-4cfb-9fd0-d89059dbb309","Type":"ContainerStarted","Data":"0257cc713afafa05f300097e4560c9a945462cd60fd7c72876a8516c89c8ed09"} Nov 04 05:36:46.304872 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:46.304797 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-hktgm"] Nov 04 05:36:46.317654 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:46.317476 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-hktgm"] Nov 04 05:36:46.317654 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:46.317615 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hktgm" Nov 04 05:36:46.323897 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:46.323867 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-mwjnl\"" Nov 04 05:36:46.437209 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:46.436998 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bc49b45-21c3-4c36-871b-157e538dea5f-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-hktgm\" (UID: \"1bc49b45-21c3-4c36-871b-157e538dea5f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hktgm" Nov 04 05:36:46.437209 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:46.437055 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtg7z\" (UniqueName: \"kubernetes.io/projected/1bc49b45-21c3-4c36-871b-157e538dea5f-kube-api-access-xtg7z\") pod \"cert-manager-cainjector-5c57bbd4fc-hktgm\" (UID: \"1bc49b45-21c3-4c36-871b-157e538dea5f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hktgm" Nov 04 05:36:46.538770 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:46.538727 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bc49b45-21c3-4c36-871b-157e538dea5f-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-hktgm\" (UID: \"1bc49b45-21c3-4c36-871b-157e538dea5f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hktgm" Nov 04 05:36:46.538975 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:46.538790 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtg7z\" (UniqueName: \"kubernetes.io/projected/1bc49b45-21c3-4c36-871b-157e538dea5f-kube-api-access-xtg7z\") pod \"cert-manager-cainjector-5c57bbd4fc-hktgm\" (UID: \"1bc49b45-21c3-4c36-871b-157e538dea5f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hktgm" Nov 04 05:36:46.547078 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:46.547048 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtg7z\" (UniqueName: \"kubernetes.io/projected/1bc49b45-21c3-4c36-871b-157e538dea5f-kube-api-access-xtg7z\") pod \"cert-manager-cainjector-5c57bbd4fc-hktgm\" (UID: \"1bc49b45-21c3-4c36-871b-157e538dea5f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hktgm" Nov 04 05:36:46.547192 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:46.547136 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bc49b45-21c3-4c36-871b-157e538dea5f-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-hktgm\" (UID: \"1bc49b45-21c3-4c36-871b-157e538dea5f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hktgm" Nov 04 05:36:46.634415 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:46.634377 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hktgm" Nov 04 05:36:46.934471 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:46.934392 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-hktgm"] Nov 04 05:36:46.938294 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:36:46.938266 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1bc49b45_21c3_4c36_871b_157e538dea5f.slice/crio-b258d66dbdf03abe48bd3fc2c09701cab3967c24ea181d7ee76dca7b254891c9 WatchSource:0}: Error finding container b258d66dbdf03abe48bd3fc2c09701cab3967c24ea181d7ee76dca7b254891c9: Status 404 returned error can't find the container with id b258d66dbdf03abe48bd3fc2c09701cab3967c24ea181d7ee76dca7b254891c9 Nov 04 05:36:46.959002 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:46.958935 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hktgm" event={"ID":"1bc49b45-21c3-4c36-871b-157e538dea5f","Type":"ContainerStarted","Data":"b258d66dbdf03abe48bd3fc2c09701cab3967c24ea181d7ee76dca7b254891c9"} Nov 04 05:36:47.968629 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:47.967780 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" event={"ID":"ac9c1355-171c-413a-9e9a-6c6d3dc92da1","Type":"ContainerStarted","Data":"484fe0245379d1095c203cf6ccbbf408a6bb8573cc1ffa023780dfe1b1ca23a3"} Nov 04 05:36:48.975105 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:48.975018 5904 generic.go:356] "Generic (PLEG): container finished" podID="ac9c1355-171c-413a-9e9a-6c6d3dc92da1" containerID="484fe0245379d1095c203cf6ccbbf408a6bb8573cc1ffa023780dfe1b1ca23a3" exitCode=0 Nov 04 05:36:48.975105 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:48.975069 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" event={"ID":"ac9c1355-171c-413a-9e9a-6c6d3dc92da1","Type":"ContainerDied","Data":"484fe0245379d1095c203cf6ccbbf408a6bb8573cc1ffa023780dfe1b1ca23a3"} Nov 04 05:36:49.981990 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:49.981949 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k2r4j" event={"ID":"323e147c-bf76-4cfb-9fd0-d89059dbb309","Type":"ContainerStarted","Data":"69eedfd04c94f38ef6a1534ccfb008f1ff3377b234199826e22be7a27c55dde0"} Nov 04 05:36:49.982453 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:49.982059 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k2r4j" Nov 04 05:36:49.983924 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:49.983902 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" event={"ID":"ac9c1355-171c-413a-9e9a-6c6d3dc92da1","Type":"ContainerStarted","Data":"e814a4f95c2773f0b62d3ff4577e1fa93e00c4dc1cd680f4116b02742ab2f503"} Nov 04 05:36:49.984063 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:49.984028 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:36:50.004170 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:50.004115 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k2r4j" podStartSLOduration=1.869065334 podStartE2EDuration="6.004098979s" totalImagesPullingTime="4.135033645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:50.002965233 +0000 UTC m=+456.960550196" watchObservedRunningTime="2026-11-04 05:36:50.004098979 +0000 UTC m=+456.961683929" Nov 04 05:36:50.039905 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:50.039859 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" podStartSLOduration=2.317706931 podStartE2EDuration="7.03984628s" totalImagesPullingTime="3.972362281s" totalInitContainerRuntime="749.777068ms" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:50.039754873 +0000 UTC m=+456.997339834" watchObservedRunningTime="2026-11-04 05:36:50.03984628 +0000 UTC m=+456.997431229" Nov 04 05:36:50.990175 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:50.990137 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hktgm" event={"ID":"1bc49b45-21c3-4c36-871b-157e538dea5f","Type":"ContainerStarted","Data":"db4fa60acfe959314be4db87f016335f5a7bfa06361d25da342f540695b1bf35"} Nov 04 05:36:51.029928 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:51.029864 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hktgm" podStartSLOduration=1.9620624 podStartE2EDuration="5.029845409s" totalImagesPullingTime="3.067783009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:51.028986453 +0000 UTC m=+457.986571402" watchObservedRunningTime="2026-11-04 05:36:51.029845409 +0000 UTC m=+457.987430353" Nov 04 05:36:55.992616 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:55.992533 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k2r4j" Nov 04 05:36:56.003322 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:56.003265 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-d86ds"] Nov 04 05:36:56.010054 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:56.009763 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-d86ds" Nov 04 05:36:56.016508 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:56.016486 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-d86ds"] Nov 04 05:36:56.018068 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:56.018047 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-42wz5\"" Nov 04 05:36:56.126999 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:56.126958 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c29rk\" (UniqueName: \"kubernetes.io/projected/b114732c-638c-46f8-b9f8-f014f7165a33-kube-api-access-c29rk\") pod \"cert-manager-597b95cb84-d86ds\" (UID: \"b114732c-638c-46f8-b9f8-f014f7165a33\") " pod="cert-manager/cert-manager-597b95cb84-d86ds" Nov 04 05:36:56.127165 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:56.127014 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b114732c-638c-46f8-b9f8-f014f7165a33-bound-sa-token\") pod \"cert-manager-597b95cb84-d86ds\" (UID: \"b114732c-638c-46f8-b9f8-f014f7165a33\") " pod="cert-manager/cert-manager-597b95cb84-d86ds" Nov 04 05:36:56.228394 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:56.228365 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c29rk\" (UniqueName: \"kubernetes.io/projected/b114732c-638c-46f8-b9f8-f014f7165a33-kube-api-access-c29rk\") pod \"cert-manager-597b95cb84-d86ds\" (UID: \"b114732c-638c-46f8-b9f8-f014f7165a33\") " pod="cert-manager/cert-manager-597b95cb84-d86ds" Nov 04 05:36:56.228545 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:56.228412 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b114732c-638c-46f8-b9f8-f014f7165a33-bound-sa-token\") pod \"cert-manager-597b95cb84-d86ds\" (UID: \"b114732c-638c-46f8-b9f8-f014f7165a33\") " pod="cert-manager/cert-manager-597b95cb84-d86ds" Nov 04 05:36:56.236627 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:56.236601 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b114732c-638c-46f8-b9f8-f014f7165a33-bound-sa-token\") pod \"cert-manager-597b95cb84-d86ds\" (UID: \"b114732c-638c-46f8-b9f8-f014f7165a33\") " pod="cert-manager/cert-manager-597b95cb84-d86ds" Nov 04 05:36:56.236732 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:56.236670 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c29rk\" (UniqueName: \"kubernetes.io/projected/b114732c-638c-46f8-b9f8-f014f7165a33-kube-api-access-c29rk\") pod \"cert-manager-597b95cb84-d86ds\" (UID: \"b114732c-638c-46f8-b9f8-f014f7165a33\") " pod="cert-manager/cert-manager-597b95cb84-d86ds" Nov 04 05:36:56.322651 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:56.322615 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-d86ds" Nov 04 05:36:56.591569 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:56.591544 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-d86ds"] Nov 04 05:36:56.594160 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:36:56.594134 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb114732c_638c_46f8_b9f8_f014f7165a33.slice/crio-a497938c6c92385a48610d2bbdd4f7ad4cb249dfdb2ffc4162c5af978d2d4608 WatchSource:0}: Error finding container a497938c6c92385a48610d2bbdd4f7ad4cb249dfdb2ffc4162c5af978d2d4608: Status 404 returned error can't find the container with id a497938c6c92385a48610d2bbdd4f7ad4cb249dfdb2ffc4162c5af978d2d4608 Nov 04 05:36:57.018057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:57.017966 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-d86ds" event={"ID":"b114732c-638c-46f8-b9f8-f014f7165a33","Type":"ContainerStarted","Data":"f2c1b644bb3d145cac3f2bcaa6ea397f27ccc0334852252882d95ce146c4782b"} Nov 04 05:36:57.018057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:57.018018 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-d86ds" event={"ID":"b114732c-638c-46f8-b9f8-f014f7165a33","Type":"ContainerStarted","Data":"a497938c6c92385a48610d2bbdd4f7ad4cb249dfdb2ffc4162c5af978d2d4608"} Nov 04 05:36:57.034137 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:57.034086 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-d86ds" podStartSLOduration=2.034071402 podStartE2EDuration="2.034071402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:57.032828284 +0000 UTC m=+463.990413230" watchObservedRunningTime="2026-11-04 05:36:57.034071402 +0000 UTC m=+463.991656347" Nov 04 05:36:57.131352 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:36:57.131317 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:36:58.837414 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:58.837379 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-cw8cs"] Nov 04 05:36:58.840730 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:58.840713 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-cw8cs" Nov 04 05:36:58.850260 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:58.850221 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:36:58.850377 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:58.850294 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\"" Nov 04 05:36:58.850732 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:58.850714 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-95dnm\"" Nov 04 05:36:58.851312 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:58.851286 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-cw8cs"] Nov 04 05:36:58.853035 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:58.853017 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vj8b\" (UniqueName: \"kubernetes.io/projected/6041142f-186a-4fef-b822-b709112015a0-kube-api-access-7vj8b\") pod \"jobset-operator-674cd59dc5-cw8cs\" (UID: \"6041142f-186a-4fef-b822-b709112015a0\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-cw8cs" Nov 04 05:36:58.853138 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:58.853044 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6041142f-186a-4fef-b822-b709112015a0-tmp\") pod \"jobset-operator-674cd59dc5-cw8cs\" (UID: \"6041142f-186a-4fef-b822-b709112015a0\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-cw8cs" Nov 04 05:36:58.953755 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:58.953717 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vj8b\" (UniqueName: \"kubernetes.io/projected/6041142f-186a-4fef-b822-b709112015a0-kube-api-access-7vj8b\") pod \"jobset-operator-674cd59dc5-cw8cs\" (UID: \"6041142f-186a-4fef-b822-b709112015a0\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-cw8cs" Nov 04 05:36:58.953755 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:58.953757 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6041142f-186a-4fef-b822-b709112015a0-tmp\") pod \"jobset-operator-674cd59dc5-cw8cs\" (UID: \"6041142f-186a-4fef-b822-b709112015a0\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-cw8cs" Nov 04 05:36:58.954146 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:58.954114 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6041142f-186a-4fef-b822-b709112015a0-tmp\") pod \"jobset-operator-674cd59dc5-cw8cs\" (UID: \"6041142f-186a-4fef-b822-b709112015a0\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-cw8cs" Nov 04 05:36:58.960999 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:58.960970 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vj8b\" (UniqueName: \"kubernetes.io/projected/6041142f-186a-4fef-b822-b709112015a0-kube-api-access-7vj8b\") pod \"jobset-operator-674cd59dc5-cw8cs\" (UID: \"6041142f-186a-4fef-b822-b709112015a0\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-cw8cs" Nov 04 05:36:59.152638 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:59.152555 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-cw8cs" Nov 04 05:36:59.424257 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:36:59.424179 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-cw8cs"] Nov 04 05:36:59.426852 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:36:59.426815 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6041142f_186a_4fef_b822_b709112015a0.slice/crio-3f410704ce7108e9c198d80e6e2a685fd7e62e8d83954b535261218a56bdbfaa WatchSource:0}: Error finding container 3f410704ce7108e9c198d80e6e2a685fd7e62e8d83954b535261218a56bdbfaa: Status 404 returned error can't find the container with id 3f410704ce7108e9c198d80e6e2a685fd7e62e8d83954b535261218a56bdbfaa Nov 04 05:37:00.033205 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:00.033169 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-cw8cs" event={"ID":"6041142f-186a-4fef-b822-b709112015a0","Type":"ContainerStarted","Data":"3f410704ce7108e9c198d80e6e2a685fd7e62e8d83954b535261218a56bdbfaa"} Nov 04 05:37:00.992532 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:00.992501 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-56cd9ffd8-ph9j8" Nov 04 05:37:02.048363 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:02.048271 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-cw8cs" event={"ID":"6041142f-186a-4fef-b822-b709112015a0","Type":"ContainerStarted","Data":"97ae16af3b0a7fd4eefd6ffba328cc466f47e85a5674464c3f45d13658a1f0fc"} Nov 04 05:37:02.066253 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:02.066179 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-cw8cs" podStartSLOduration=1.92592697 podStartE2EDuration="4.066164365s" totalImagesPullingTime="2.140237395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:37:02.064109096 +0000 UTC m=+469.021694053" watchObservedRunningTime="2026-11-04 05:37:02.066164365 +0000 UTC m=+469.023749315" Nov 04 05:37:15.070421 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.070377 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6"] Nov 04 05:37:15.074041 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.074017 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6" Nov 04 05:37:15.085806 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.085771 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Nov 04 05:37:15.086139 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.086103 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Nov 04 05:37:15.088426 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.086751 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-fm4tq\"" Nov 04 05:37:15.089410 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.089384 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Nov 04 05:37:15.089669 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.089652 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Nov 04 05:37:15.091628 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.091604 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6"] Nov 04 05:37:15.198414 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.198381 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5548164-9095-44ae-820c-dbddc724ecbf-tls-certs\") pod \"kube-auth-proxy-5488c58ff5-nc2z6\" (UID: \"b5548164-9095-44ae-820c-dbddc724ecbf\") " pod="openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6" Nov 04 05:37:15.198603 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.198541 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhrkf\" (UniqueName: \"kubernetes.io/projected/b5548164-9095-44ae-820c-dbddc724ecbf-kube-api-access-hhrkf\") pod \"kube-auth-proxy-5488c58ff5-nc2z6\" (UID: \"b5548164-9095-44ae-820c-dbddc724ecbf\") " pod="openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6" Nov 04 05:37:15.198666 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.198616 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5548164-9095-44ae-820c-dbddc724ecbf-tmp\") pod \"kube-auth-proxy-5488c58ff5-nc2z6\" (UID: \"b5548164-9095-44ae-820c-dbddc724ecbf\") " pod="openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6" Nov 04 05:37:15.299280 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.299240 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5548164-9095-44ae-820c-dbddc724ecbf-tmp\") pod \"kube-auth-proxy-5488c58ff5-nc2z6\" (UID: \"b5548164-9095-44ae-820c-dbddc724ecbf\") " pod="openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6" Nov 04 05:37:15.299478 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.299322 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5548164-9095-44ae-820c-dbddc724ecbf-tls-certs\") pod \"kube-auth-proxy-5488c58ff5-nc2z6\" (UID: \"b5548164-9095-44ae-820c-dbddc724ecbf\") " pod="openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6" Nov 04 05:37:15.299478 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.299423 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhrkf\" (UniqueName: \"kubernetes.io/projected/b5548164-9095-44ae-820c-dbddc724ecbf-kube-api-access-hhrkf\") pod \"kube-auth-proxy-5488c58ff5-nc2z6\" (UID: \"b5548164-9095-44ae-820c-dbddc724ecbf\") " pod="openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6" Nov 04 05:37:15.302009 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.301986 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5548164-9095-44ae-820c-dbddc724ecbf-tls-certs\") pod \"kube-auth-proxy-5488c58ff5-nc2z6\" (UID: \"b5548164-9095-44ae-820c-dbddc724ecbf\") " pod="openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6" Nov 04 05:37:15.302959 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.302935 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5548164-9095-44ae-820c-dbddc724ecbf-tmp\") pod \"kube-auth-proxy-5488c58ff5-nc2z6\" (UID: \"b5548164-9095-44ae-820c-dbddc724ecbf\") " pod="openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6" Nov 04 05:37:15.307338 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.307311 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhrkf\" (UniqueName: \"kubernetes.io/projected/b5548164-9095-44ae-820c-dbddc724ecbf-kube-api-access-hhrkf\") pod \"kube-auth-proxy-5488c58ff5-nc2z6\" (UID: \"b5548164-9095-44ae-820c-dbddc724ecbf\") " pod="openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6" Nov 04 05:37:15.392039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.391958 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6" Nov 04 05:37:15.663829 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:15.663798 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6"] Nov 04 05:37:15.665790 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:37:15.665756 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5548164_9095_44ae_820c_dbddc724ecbf.slice/crio-0d42c29c884fa08d53e39f98b65876ca3722acb318ae387cedca662c9534c43a WatchSource:0}: Error finding container 0d42c29c884fa08d53e39f98b65876ca3722acb318ae387cedca662c9534c43a: Status 404 returned error can't find the container with id 0d42c29c884fa08d53e39f98b65876ca3722acb318ae387cedca662c9534c43a Nov 04 05:37:16.055584 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.055507 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-fdm8x"] Nov 04 05:37:16.058438 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.058420 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" Nov 04 05:37:16.069077 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.069052 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-nldtc\"" Nov 04 05:37:16.070037 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.070011 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Nov 04 05:37:16.070338 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.070325 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Nov 04 05:37:16.070409 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.070339 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Nov 04 05:37:16.071555 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.071537 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-fdm8x"] Nov 04 05:37:16.106415 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.106386 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/21e29ad4-921a-437c-bfd3-ea5ab96db4cf-config-file\") pod \"limitador-limitador-74dfcfb894-fdm8x\" (UID: \"21e29ad4-921a-437c-bfd3-ea5ab96db4cf\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" Nov 04 05:37:16.106582 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.106428 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxjqn\" (UniqueName: \"kubernetes.io/projected/21e29ad4-921a-437c-bfd3-ea5ab96db4cf-kube-api-access-gxjqn\") pod \"limitador-limitador-74dfcfb894-fdm8x\" (UID: \"21e29ad4-921a-437c-bfd3-ea5ab96db4cf\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" Nov 04 05:37:16.112800 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.112772 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6" event={"ID":"b5548164-9095-44ae-820c-dbddc724ecbf","Type":"ContainerStarted","Data":"0d42c29c884fa08d53e39f98b65876ca3722acb318ae387cedca662c9534c43a"} Nov 04 05:37:16.161735 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.161705 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-fdm8x"] Nov 04 05:37:16.207241 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.207190 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/21e29ad4-921a-437c-bfd3-ea5ab96db4cf-config-file\") pod \"limitador-limitador-74dfcfb894-fdm8x\" (UID: \"21e29ad4-921a-437c-bfd3-ea5ab96db4cf\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" Nov 04 05:37:16.207413 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.207391 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxjqn\" (UniqueName: \"kubernetes.io/projected/21e29ad4-921a-437c-bfd3-ea5ab96db4cf-kube-api-access-gxjqn\") pod \"limitador-limitador-74dfcfb894-fdm8x\" (UID: \"21e29ad4-921a-437c-bfd3-ea5ab96db4cf\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" Nov 04 05:37:16.207803 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.207785 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/21e29ad4-921a-437c-bfd3-ea5ab96db4cf-config-file\") pod \"limitador-limitador-74dfcfb894-fdm8x\" (UID: \"21e29ad4-921a-437c-bfd3-ea5ab96db4cf\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" Nov 04 05:37:16.221987 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.221959 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxjqn\" (UniqueName: \"kubernetes.io/projected/21e29ad4-921a-437c-bfd3-ea5ab96db4cf-kube-api-access-gxjqn\") pod \"limitador-limitador-74dfcfb894-fdm8x\" (UID: \"21e29ad4-921a-437c-bfd3-ea5ab96db4cf\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" Nov 04 05:37:16.374153 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.372987 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" Nov 04 05:37:16.668958 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.668918 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-fdm8x"] Nov 04 05:37:16.672190 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:37:16.672160 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21e29ad4_921a_437c_bfd3_ea5ab96db4cf.slice/crio-7562f9ce2ff3759984e844c92a2d71ad516c53ea0edafaba5a7de17b8d9257db WatchSource:0}: Error finding container 7562f9ce2ff3759984e844c92a2d71ad516c53ea0edafaba5a7de17b8d9257db: Status 404 returned error can't find the container with id 7562f9ce2ff3759984e844c92a2d71ad516c53ea0edafaba5a7de17b8d9257db Nov 04 05:37:16.698525 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.698498 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh"] Nov 04 05:37:16.701692 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.701672 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh" Nov 04 05:37:16.711698 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.710296 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Nov 04 05:37:16.711698 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.711404 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh"] Nov 04 05:37:16.731864 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.731836 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Nov 04 05:37:16.813280 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.813237 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f31c8bc8-3733-4b18-82fb-a7273a6300df-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-wm6vh\" (UID: \"f31c8bc8-3733-4b18-82fb-a7273a6300df\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh" Nov 04 05:37:16.813466 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.813299 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f31c8bc8-3733-4b18-82fb-a7273a6300df-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-wm6vh\" (UID: \"f31c8bc8-3733-4b18-82fb-a7273a6300df\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh" Nov 04 05:37:16.813555 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.813527 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxjv8\" (UniqueName: \"kubernetes.io/projected/f31c8bc8-3733-4b18-82fb-a7273a6300df-kube-api-access-mxjv8\") pod \"kuadrant-console-plugin-66c4fb575d-wm6vh\" (UID: \"f31c8bc8-3733-4b18-82fb-a7273a6300df\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh" Nov 04 05:37:16.914937 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.914895 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxjv8\" (UniqueName: \"kubernetes.io/projected/f31c8bc8-3733-4b18-82fb-a7273a6300df-kube-api-access-mxjv8\") pod \"kuadrant-console-plugin-66c4fb575d-wm6vh\" (UID: \"f31c8bc8-3733-4b18-82fb-a7273a6300df\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh" Nov 04 05:37:16.915108 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.914986 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f31c8bc8-3733-4b18-82fb-a7273a6300df-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-wm6vh\" (UID: \"f31c8bc8-3733-4b18-82fb-a7273a6300df\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh" Nov 04 05:37:16.915108 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.915022 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f31c8bc8-3733-4b18-82fb-a7273a6300df-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-wm6vh\" (UID: \"f31c8bc8-3733-4b18-82fb-a7273a6300df\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh" Nov 04 05:37:16.916186 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.916158 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f31c8bc8-3733-4b18-82fb-a7273a6300df-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-wm6vh\" (UID: \"f31c8bc8-3733-4b18-82fb-a7273a6300df\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh" Nov 04 05:37:16.917785 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.917758 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f31c8bc8-3733-4b18-82fb-a7273a6300df-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-wm6vh\" (UID: \"f31c8bc8-3733-4b18-82fb-a7273a6300df\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh" Nov 04 05:37:16.926095 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:16.926022 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxjv8\" (UniqueName: \"kubernetes.io/projected/f31c8bc8-3733-4b18-82fb-a7273a6300df-kube-api-access-mxjv8\") pod \"kuadrant-console-plugin-66c4fb575d-wm6vh\" (UID: \"f31c8bc8-3733-4b18-82fb-a7273a6300df\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh" Nov 04 05:37:17.015869 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:17.015829 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh" Nov 04 05:37:17.119138 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:17.119101 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" event={"ID":"21e29ad4-921a-437c-bfd3-ea5ab96db4cf","Type":"ContainerStarted","Data":"7562f9ce2ff3759984e844c92a2d71ad516c53ea0edafaba5a7de17b8d9257db"} Nov 04 05:37:17.294788 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:17.294692 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh"] Nov 04 05:37:17.298560 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:37:17.298524 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf31c8bc8_3733_4b18_82fb_a7273a6300df.slice/crio-4bb4766a85b642ae95b410bd4b11d60aebc445f5a36f6ab78279356bdafd16ff WatchSource:0}: Error finding container 4bb4766a85b642ae95b410bd4b11d60aebc445f5a36f6ab78279356bdafd16ff: Status 404 returned error can't find the container with id 4bb4766a85b642ae95b410bd4b11d60aebc445f5a36f6ab78279356bdafd16ff Nov 04 05:37:18.126516 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.126475 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh" event={"ID":"f31c8bc8-3733-4b18-82fb-a7273a6300df","Type":"ContainerStarted","Data":"4bb4766a85b642ae95b410bd4b11d60aebc445f5a36f6ab78279356bdafd16ff"} Nov 04 05:37:18.422623 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.421765 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc"] Nov 04 05:37:18.425369 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.425344 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" Nov 04 05:37:18.429632 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.429606 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Nov 04 05:37:18.429762 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.429628 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Nov 04 05:37:18.434745 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.434721 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\"" Nov 04 05:37:18.440965 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.440942 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc"] Nov 04 05:37:18.492920 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.492889 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-vcjbd\"" Nov 04 05:37:18.530499 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.530459 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c46526a-a778-4be1-bdf7-fc3c1f7d38b2-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-nrwnc\" (UID: \"4c46526a-a778-4be1-bdf7-fc3c1f7d38b2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" Nov 04 05:37:18.530672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.530528 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4c46526a-a778-4be1-bdf7-fc3c1f7d38b2-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-nrwnc\" (UID: \"4c46526a-a778-4be1-bdf7-fc3c1f7d38b2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" Nov 04 05:37:18.530672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.530576 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l485l\" (UniqueName: \"kubernetes.io/projected/4c46526a-a778-4be1-bdf7-fc3c1f7d38b2-kube-api-access-l485l\") pod \"kserve-module-controller-manager-7566cf5fc8-nrwnc\" (UID: \"4c46526a-a778-4be1-bdf7-fc3c1f7d38b2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" Nov 04 05:37:18.530789 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.530684 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c46526a-a778-4be1-bdf7-fc3c1f7d38b2-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-nrwnc\" (UID: \"4c46526a-a778-4be1-bdf7-fc3c1f7d38b2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" Nov 04 05:37:18.631741 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.631695 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c46526a-a778-4be1-bdf7-fc3c1f7d38b2-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-nrwnc\" (UID: \"4c46526a-a778-4be1-bdf7-fc3c1f7d38b2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" Nov 04 05:37:18.631932 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.631764 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4c46526a-a778-4be1-bdf7-fc3c1f7d38b2-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-nrwnc\" (UID: \"4c46526a-a778-4be1-bdf7-fc3c1f7d38b2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" Nov 04 05:37:18.631932 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.631801 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l485l\" (UniqueName: \"kubernetes.io/projected/4c46526a-a778-4be1-bdf7-fc3c1f7d38b2-kube-api-access-l485l\") pod \"kserve-module-controller-manager-7566cf5fc8-nrwnc\" (UID: \"4c46526a-a778-4be1-bdf7-fc3c1f7d38b2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" Nov 04 05:37:18.631932 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.631881 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c46526a-a778-4be1-bdf7-fc3c1f7d38b2-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-nrwnc\" (UID: \"4c46526a-a778-4be1-bdf7-fc3c1f7d38b2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" Nov 04 05:37:18.634600 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.634545 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c46526a-a778-4be1-bdf7-fc3c1f7d38b2-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-nrwnc\" (UID: \"4c46526a-a778-4be1-bdf7-fc3c1f7d38b2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" Nov 04 05:37:18.635131 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.635087 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4c46526a-a778-4be1-bdf7-fc3c1f7d38b2-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-nrwnc\" (UID: \"4c46526a-a778-4be1-bdf7-fc3c1f7d38b2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" Nov 04 05:37:18.635131 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.635102 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c46526a-a778-4be1-bdf7-fc3c1f7d38b2-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-nrwnc\" (UID: \"4c46526a-a778-4be1-bdf7-fc3c1f7d38b2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" Nov 04 05:37:18.641836 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.641809 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l485l\" (UniqueName: \"kubernetes.io/projected/4c46526a-a778-4be1-bdf7-fc3c1f7d38b2-kube-api-access-l485l\") pod \"kserve-module-controller-manager-7566cf5fc8-nrwnc\" (UID: \"4c46526a-a778-4be1-bdf7-fc3c1f7d38b2\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" Nov 04 05:37:18.745869 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:18.745785 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" Nov 04 05:37:19.032606 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:19.032524 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc"] Nov 04 05:37:19.036502 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:37:19.036471 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c46526a_a778_4be1_bdf7_fc3c1f7d38b2.slice/crio-ab7749488fd480fd9eae09fe35614e63cb43540b31d70539aef85bf666863a98 WatchSource:0}: Error finding container ab7749488fd480fd9eae09fe35614e63cb43540b31d70539aef85bf666863a98: Status 404 returned error can't find the container with id ab7749488fd480fd9eae09fe35614e63cb43540b31d70539aef85bf666863a98 Nov 04 05:37:19.132808 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:19.132770 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" event={"ID":"4c46526a-a778-4be1-bdf7-fc3c1f7d38b2","Type":"ContainerStarted","Data":"ab7749488fd480fd9eae09fe35614e63cb43540b31d70539aef85bf666863a98"} Nov 04 05:37:19.134734 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:19.134684 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6" event={"ID":"b5548164-9095-44ae-820c-dbddc724ecbf","Type":"ContainerStarted","Data":"daa62b2e71b0e01ae3365178d84c7d725dc9057644ef0f950765eecadcf4716d"} Nov 04 05:37:19.151528 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:19.151462 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5488c58ff5-nc2z6" podStartSLOduration=1.613392028 podStartE2EDuration="4.151443253s" totalImagesPullingTime="2.538051225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:37:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:37:19.148784895 +0000 UTC m=+486.106369854" watchObservedRunningTime="2026-11-04 05:37:19.151443253 +0000 UTC m=+486.109028205" Nov 04 05:37:22.152214 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:22.152169 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" event={"ID":"4c46526a-a778-4be1-bdf7-fc3c1f7d38b2","Type":"ContainerStarted","Data":"54d8982394687714a2297371483176819b87852e28640382a5d4e861a7626c7b"} Nov 04 05:37:22.152709 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:22.152374 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" Nov 04 05:37:23.157988 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:23.157954 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh" event={"ID":"f31c8bc8-3733-4b18-82fb-a7273a6300df","Type":"ContainerStarted","Data":"218cedfdf82219726421e8eda6f538a68f2b93aca7583d5be22ec49804bf6d3e"} Nov 04 05:37:23.172554 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:23.172253 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-wm6vh" podStartSLOduration=1.909710042 podStartE2EDuration="7.172211489s" totalImagesPullingTime="5.262501447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:37:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:37:23.171862829 +0000 UTC m=+490.129447782" watchObservedRunningTime="2026-11-04 05:37:23.172211489 +0000 UTC m=+490.129796750" Nov 04 05:37:23.172745 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:23.172562 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" podStartSLOduration=2.520262905 podStartE2EDuration="5.172552004s" totalImagesPullingTime="2.652289099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:37:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:37:22.170305026 +0000 UTC m=+489.127889979" watchObservedRunningTime="2026-11-04 05:37:23.172552004 +0000 UTC m=+490.130136960" Nov 04 05:37:27.137041 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:37:27.136999 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:37:29.187517 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:29.187485 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" event={"ID":"21e29ad4-921a-437c-bfd3-ea5ab96db4cf","Type":"ContainerStarted","Data":"0404ce936934e169c123e10355c9ff67e14a196a68261fb6eb7f0b548b9efe8f"} Nov 04 05:37:29.187939 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:29.187619 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" Nov 04 05:37:29.208919 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:29.208875 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" podStartSLOduration=1.21899637 podStartE2EDuration="13.20886252s" totalImagesPullingTime="11.98986615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:37:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:37:29.208514099 +0000 UTC m=+496.166099049" watchObservedRunningTime="2026-11-04 05:37:29.20886252 +0000 UTC m=+496.166447516" Nov 04 05:37:30.742285 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:30.742255 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-fdm8x"] Nov 04 05:37:31.196328 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:31.196289 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" podUID="21e29ad4-921a-437c-bfd3-ea5ab96db4cf" containerName="limitador" containerID="cri-o://0404ce936934e169c123e10355c9ff67e14a196a68261fb6eb7f0b548b9efe8f" gracePeriod=30 Nov 04 05:37:31.750816 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:31.750789 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" Nov 04 05:37:31.877136 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:31.877094 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21e29ad4-921a-437c-bfd3-ea5ab96db4cf-kube-api-access-gxjqn\" (UniqueName: \"kubernetes.io/projected/21e29ad4-921a-437c-bfd3-ea5ab96db4cf-kube-api-access-gxjqn\") pod \"21e29ad4-921a-437c-bfd3-ea5ab96db4cf\" (UID: \"21e29ad4-921a-437c-bfd3-ea5ab96db4cf\") " Nov 04 05:37:31.877370 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:31.877206 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21e29ad4-921a-437c-bfd3-ea5ab96db4cf-config-file\" (UniqueName: \"kubernetes.io/configmap/21e29ad4-921a-437c-bfd3-ea5ab96db4cf-config-file\") pod \"21e29ad4-921a-437c-bfd3-ea5ab96db4cf\" (UID: \"21e29ad4-921a-437c-bfd3-ea5ab96db4cf\") " Nov 04 05:37:31.877584 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:31.877549 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21e29ad4-921a-437c-bfd3-ea5ab96db4cf-config-file" pod "21e29ad4-921a-437c-bfd3-ea5ab96db4cf" (UID: "21e29ad4-921a-437c-bfd3-ea5ab96db4cf"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:37:31.879449 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:31.879421 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21e29ad4-921a-437c-bfd3-ea5ab96db4cf-kube-api-access-gxjqn" pod "21e29ad4-921a-437c-bfd3-ea5ab96db4cf" (UID: "21e29ad4-921a-437c-bfd3-ea5ab96db4cf"). InnerVolumeSpecName "kube-api-access-gxjqn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:37:31.978812 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:31.978770 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxjqn\" (UniqueName: \"kubernetes.io/projected/21e29ad4-921a-437c-bfd3-ea5ab96db4cf-kube-api-access-gxjqn\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:37:31.978812 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:31.978804 5904 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/21e29ad4-921a-437c-bfd3-ea5ab96db4cf-config-file\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:37:32.202266 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.202164 5904 generic.go:356] "Generic (PLEG): container finished" podID="21e29ad4-921a-437c-bfd3-ea5ab96db4cf" containerID="0404ce936934e169c123e10355c9ff67e14a196a68261fb6eb7f0b548b9efe8f" exitCode=0 Nov 04 05:37:32.202266 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.202207 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" Nov 04 05:37:32.202502 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.202254 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" event={"ID":"21e29ad4-921a-437c-bfd3-ea5ab96db4cf","Type":"ContainerDied","Data":"0404ce936934e169c123e10355c9ff67e14a196a68261fb6eb7f0b548b9efe8f"} Nov 04 05:37:32.202502 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.202322 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-fdm8x" event={"ID":"21e29ad4-921a-437c-bfd3-ea5ab96db4cf","Type":"ContainerDied","Data":"7562f9ce2ff3759984e844c92a2d71ad516c53ea0edafaba5a7de17b8d9257db"} Nov 04 05:37:32.202502 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.202352 5904 scope.go:122] "RemoveContainer" containerID="0404ce936934e169c123e10355c9ff67e14a196a68261fb6eb7f0b548b9efe8f" Nov 04 05:37:32.211148 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.211128 5904 scope.go:122] "RemoveContainer" containerID="0404ce936934e169c123e10355c9ff67e14a196a68261fb6eb7f0b548b9efe8f" Nov 04 05:37:32.211436 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:37:32.211418 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0404ce936934e169c123e10355c9ff67e14a196a68261fb6eb7f0b548b9efe8f\": container with ID starting with 0404ce936934e169c123e10355c9ff67e14a196a68261fb6eb7f0b548b9efe8f not found: ID does not exist" containerID="0404ce936934e169c123e10355c9ff67e14a196a68261fb6eb7f0b548b9efe8f" Nov 04 05:37:32.211497 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.211445 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0404ce936934e169c123e10355c9ff67e14a196a68261fb6eb7f0b548b9efe8f"} err="failed to get container status \"0404ce936934e169c123e10355c9ff67e14a196a68261fb6eb7f0b548b9efe8f\": rpc error: code = NotFound desc = could not find container \"0404ce936934e169c123e10355c9ff67e14a196a68261fb6eb7f0b548b9efe8f\": container with ID starting with 0404ce936934e169c123e10355c9ff67e14a196a68261fb6eb7f0b548b9efe8f not found: ID does not exist" Nov 04 05:37:32.222634 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.222611 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-fdm8x"] Nov 04 05:37:32.227631 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.227612 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-fdm8x"] Nov 04 05:37:32.788491 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.788458 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-2bpbj"] Nov 04 05:37:32.788858 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.788835 5904 cpu_manager.go:414] "containerMap: removing container" podUID="21e29ad4-921a-437c-bfd3-ea5ab96db4cf" containerName="limitador" Nov 04 05:37:32.788858 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.788848 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21e29ad4-921a-437c-bfd3-ea5ab96db4cf" containerName="limitador" Nov 04 05:37:32.788944 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.788919 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="21e29ad4-921a-437c-bfd3-ea5ab96db4cf" containerName="limitador" Nov 04 05:37:32.792722 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.792704 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2bpbj" Nov 04 05:37:32.799879 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.799853 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 05:37:32.799999 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.799913 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 05:37:32.800055 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.799999 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-2fxlq\"" Nov 04 05:37:32.800097 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.800068 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Nov 04 05:37:32.800895 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.800875 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-2bpbj"] Nov 04 05:37:32.987986 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.987953 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2b8a322c-d8e3-4457-bce3-bf609408418e-data\") pod \"postgres-7cdbf6dc44-2bpbj\" (UID: \"2b8a322c-d8e3-4457-bce3-bf609408418e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2bpbj" Nov 04 05:37:32.987986 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:32.987987 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cvbl\" (UniqueName: \"kubernetes.io/projected/2b8a322c-d8e3-4457-bce3-bf609408418e-kube-api-access-4cvbl\") pod \"postgres-7cdbf6dc44-2bpbj\" (UID: \"2b8a322c-d8e3-4457-bce3-bf609408418e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2bpbj" Nov 04 05:37:33.089150 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:33.089119 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2b8a322c-d8e3-4457-bce3-bf609408418e-data\") pod \"postgres-7cdbf6dc44-2bpbj\" (UID: \"2b8a322c-d8e3-4457-bce3-bf609408418e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2bpbj" Nov 04 05:37:33.089337 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:33.089155 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cvbl\" (UniqueName: \"kubernetes.io/projected/2b8a322c-d8e3-4457-bce3-bf609408418e-kube-api-access-4cvbl\") pod \"postgres-7cdbf6dc44-2bpbj\" (UID: \"2b8a322c-d8e3-4457-bce3-bf609408418e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2bpbj" Nov 04 05:37:33.089527 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:33.089506 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2b8a322c-d8e3-4457-bce3-bf609408418e-data\") pod \"postgres-7cdbf6dc44-2bpbj\" (UID: \"2b8a322c-d8e3-4457-bce3-bf609408418e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2bpbj" Nov 04 05:37:33.103674 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:33.103642 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cvbl\" (UniqueName: \"kubernetes.io/projected/2b8a322c-d8e3-4457-bce3-bf609408418e-kube-api-access-4cvbl\") pod \"postgres-7cdbf6dc44-2bpbj\" (UID: \"2b8a322c-d8e3-4457-bce3-bf609408418e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2bpbj" Nov 04 05:37:33.107480 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:33.107462 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2bpbj" Nov 04 05:37:33.174750 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:33.174714 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21e29ad4-921a-437c-bfd3-ea5ab96db4cf" path="/var/lib/kubelet/pods/21e29ad4-921a-437c-bfd3-ea5ab96db4cf/volumes" Nov 04 05:37:33.378268 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:33.378246 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-2bpbj"] Nov 04 05:37:33.380500 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:37:33.380475 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b8a322c_d8e3_4457_bce3_bf609408418e.slice/crio-16f60d5a1a69ccace0c1fd50e11de564b9c30503aef2b8e2346c84445c9f6ca3 WatchSource:0}: Error finding container 16f60d5a1a69ccace0c1fd50e11de564b9c30503aef2b8e2346c84445c9f6ca3: Status 404 returned error can't find the container with id 16f60d5a1a69ccace0c1fd50e11de564b9c30503aef2b8e2346c84445c9f6ca3 Nov 04 05:37:34.213276 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:34.213243 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2bpbj" event={"ID":"2b8a322c-d8e3-4457-bce3-bf609408418e","Type":"ContainerStarted","Data":"16f60d5a1a69ccace0c1fd50e11de564b9c30503aef2b8e2346c84445c9f6ca3"} Nov 04 05:37:43.159999 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:43.159335 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-nrwnc" Nov 04 05:37:47.279139 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:47.279091 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2bpbj" event={"ID":"2b8a322c-d8e3-4457-bce3-bf609408418e","Type":"ContainerStarted","Data":"1323a3d965cbe7208a1de5522dfd0f604c4600250b169c310f43d16a6a43ee45"} Nov 04 05:37:47.279591 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:47.279322 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2bpbj" Nov 04 05:37:47.294572 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:47.294506 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2bpbj" podStartSLOduration=1.9627052759999999 podStartE2EDuration="15.294492381s" totalImagesPullingTime="13.331787105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:37:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:37:47.294187233 +0000 UTC m=+514.251772190" watchObservedRunningTime="2026-11-04 05:37:47.294492381 +0000 UTC m=+514.252077374" Nov 04 05:37:53.313770 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:37:53.313739 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2bpbj" Nov 04 05:37:57.131933 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:37:57.131895 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:38:00.888521 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:00.888489 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-595689b685-gmspv"] Nov 04 05:38:00.891863 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:00.891846 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-595689b685-gmspv" Nov 04 05:38:00.898413 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:00.898302 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Nov 04 05:38:00.900312 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:00.900284 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Nov 04 05:38:00.900418 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:00.900379 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Nov 04 05:38:00.900467 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:00.900452 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4tnqg\"" Nov 04 05:38:00.902300 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:00.902278 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-595689b685-gmspv"] Nov 04 05:38:01.049605 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:01.049564 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57c93381-b98d-4d00-baf7-1b31d0f6c433-webhook-cert\") pod \"maas-controller-595689b685-gmspv\" (UID: \"57c93381-b98d-4d00-baf7-1b31d0f6c433\") " pod="opendatahub/maas-controller-595689b685-gmspv" Nov 04 05:38:01.049605 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:01.049601 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57c93381-b98d-4d00-baf7-1b31d0f6c433-metrics-certs\") pod \"maas-controller-595689b685-gmspv\" (UID: \"57c93381-b98d-4d00-baf7-1b31d0f6c433\") " pod="opendatahub/maas-controller-595689b685-gmspv" Nov 04 05:38:01.049910 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:01.049694 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm2zt\" (UniqueName: \"kubernetes.io/projected/57c93381-b98d-4d00-baf7-1b31d0f6c433-kube-api-access-hm2zt\") pod \"maas-controller-595689b685-gmspv\" (UID: \"57c93381-b98d-4d00-baf7-1b31d0f6c433\") " pod="opendatahub/maas-controller-595689b685-gmspv" Nov 04 05:38:01.151069 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:01.150949 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57c93381-b98d-4d00-baf7-1b31d0f6c433-webhook-cert\") pod \"maas-controller-595689b685-gmspv\" (UID: \"57c93381-b98d-4d00-baf7-1b31d0f6c433\") " pod="opendatahub/maas-controller-595689b685-gmspv" Nov 04 05:38:01.151069 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:01.151045 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57c93381-b98d-4d00-baf7-1b31d0f6c433-metrics-certs\") pod \"maas-controller-595689b685-gmspv\" (UID: \"57c93381-b98d-4d00-baf7-1b31d0f6c433\") " pod="opendatahub/maas-controller-595689b685-gmspv" Nov 04 05:38:01.151361 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:01.151113 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm2zt\" (UniqueName: \"kubernetes.io/projected/57c93381-b98d-4d00-baf7-1b31d0f6c433-kube-api-access-hm2zt\") pod \"maas-controller-595689b685-gmspv\" (UID: \"57c93381-b98d-4d00-baf7-1b31d0f6c433\") " pod="opendatahub/maas-controller-595689b685-gmspv" Nov 04 05:38:01.153575 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:01.153548 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57c93381-b98d-4d00-baf7-1b31d0f6c433-metrics-certs\") pod \"maas-controller-595689b685-gmspv\" (UID: \"57c93381-b98d-4d00-baf7-1b31d0f6c433\") " pod="opendatahub/maas-controller-595689b685-gmspv" Nov 04 05:38:01.153682 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:01.153574 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57c93381-b98d-4d00-baf7-1b31d0f6c433-webhook-cert\") pod \"maas-controller-595689b685-gmspv\" (UID: \"57c93381-b98d-4d00-baf7-1b31d0f6c433\") " pod="opendatahub/maas-controller-595689b685-gmspv" Nov 04 05:38:01.160508 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:01.160483 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm2zt\" (UniqueName: \"kubernetes.io/projected/57c93381-b98d-4d00-baf7-1b31d0f6c433-kube-api-access-hm2zt\") pod \"maas-controller-595689b685-gmspv\" (UID: \"57c93381-b98d-4d00-baf7-1b31d0f6c433\") " pod="opendatahub/maas-controller-595689b685-gmspv" Nov 04 05:38:01.207720 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:01.207679 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-595689b685-gmspv" Nov 04 05:38:01.485518 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:01.485484 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-595689b685-gmspv"] Nov 04 05:38:01.488005 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:38:01.487977 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57c93381_b98d_4d00_baf7_1b31d0f6c433.slice/crio-763d09ef1231e129eac8dbeef90c9ee7f5ed4d6cb68f75a253bc03b0367c323e WatchSource:0}: Error finding container 763d09ef1231e129eac8dbeef90c9ee7f5ed4d6cb68f75a253bc03b0367c323e: Status 404 returned error can't find the container with id 763d09ef1231e129eac8dbeef90c9ee7f5ed4d6cb68f75a253bc03b0367c323e Nov 04 05:38:02.347668 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:02.347627 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-595689b685-gmspv" event={"ID":"57c93381-b98d-4d00-baf7-1b31d0f6c433","Type":"ContainerStarted","Data":"763d09ef1231e129eac8dbeef90c9ee7f5ed4d6cb68f75a253bc03b0367c323e"} Nov 04 05:38:04.371989 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:04.371897 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-595689b685-gmspv" event={"ID":"57c93381-b98d-4d00-baf7-1b31d0f6c433","Type":"ContainerStarted","Data":"06b1c6823e4ce479278c2fe437af528f6e24f81be77acfe927bb4a12ba2a69a8"} Nov 04 05:38:04.372373 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:04.371994 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-595689b685-gmspv" Nov 04 05:38:04.389536 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:04.389483 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-595689b685-gmspv" podStartSLOduration=1.804680114 podStartE2EDuration="4.389469024s" totalImagesPullingTime="2.58478891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:04.388913554 +0000 UTC m=+531.346498506" watchObservedRunningTime="2026-11-04 05:38:04.389469024 +0000 UTC m=+531.347053974" Nov 04 05:38:04.866869 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:04.866837 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm"] Nov 04 05:38:04.870879 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:04.870847 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" Nov 04 05:38:04.878270 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:04.877997 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\"" Nov 04 05:38:04.879379 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:04.879006 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\"" Nov 04 05:38:04.879379 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:04.879098 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-4hchn\"" Nov 04 05:38:04.879379 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:04.879141 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\"" Nov 04 05:38:04.880169 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:04.880112 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm"] Nov 04 05:38:04.985529 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:04.985498 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/36073709-5d75-4ba9-9adf-fb35c540843b-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mkgsm\" (UID: \"36073709-5d75-4ba9-9adf-fb35c540843b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" Nov 04 05:38:04.985711 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:04.985624 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/36073709-5d75-4ba9-9adf-fb35c540843b-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mkgsm\" (UID: \"36073709-5d75-4ba9-9adf-fb35c540843b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" Nov 04 05:38:04.985711 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:04.985661 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36073709-5d75-4ba9-9adf-fb35c540843b-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mkgsm\" (UID: \"36073709-5d75-4ba9-9adf-fb35c540843b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" Nov 04 05:38:04.985784 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:04.985728 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22zrs\" (UniqueName: \"kubernetes.io/projected/36073709-5d75-4ba9-9adf-fb35c540843b-kube-api-access-22zrs\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mkgsm\" (UID: \"36073709-5d75-4ba9-9adf-fb35c540843b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" Nov 04 05:38:05.087065 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:05.087030 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36073709-5d75-4ba9-9adf-fb35c540843b-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mkgsm\" (UID: \"36073709-5d75-4ba9-9adf-fb35c540843b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" Nov 04 05:38:05.087065 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:05.087065 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22zrs\" (UniqueName: \"kubernetes.io/projected/36073709-5d75-4ba9-9adf-fb35c540843b-kube-api-access-22zrs\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mkgsm\" (UID: \"36073709-5d75-4ba9-9adf-fb35c540843b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" Nov 04 05:38:05.087336 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:05.087099 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/36073709-5d75-4ba9-9adf-fb35c540843b-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mkgsm\" (UID: \"36073709-5d75-4ba9-9adf-fb35c540843b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" Nov 04 05:38:05.087336 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:05.087205 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/36073709-5d75-4ba9-9adf-fb35c540843b-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mkgsm\" (UID: \"36073709-5d75-4ba9-9adf-fb35c540843b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" Nov 04 05:38:05.087933 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:05.087902 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36073709-5d75-4ba9-9adf-fb35c540843b-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mkgsm\" (UID: \"36073709-5d75-4ba9-9adf-fb35c540843b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" Nov 04 05:38:05.088168 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:05.088151 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/36073709-5d75-4ba9-9adf-fb35c540843b-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mkgsm\" (UID: \"36073709-5d75-4ba9-9adf-fb35c540843b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" Nov 04 05:38:05.089570 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:05.089548 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/36073709-5d75-4ba9-9adf-fb35c540843b-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mkgsm\" (UID: \"36073709-5d75-4ba9-9adf-fb35c540843b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" Nov 04 05:38:05.094703 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:05.094680 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22zrs\" (UniqueName: \"kubernetes.io/projected/36073709-5d75-4ba9-9adf-fb35c540843b-kube-api-access-22zrs\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-mkgsm\" (UID: \"36073709-5d75-4ba9-9adf-fb35c540843b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" Nov 04 05:38:05.187899 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:05.187826 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" Nov 04 05:38:05.477436 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:05.477355 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm"] Nov 04 05:38:05.480334 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:38:05.480307 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36073709_5d75_4ba9_9adf_fb35c540843b.slice/crio-51d09befa06958f5176aacfae3f4fd107a9b0a3b577ce2436f8b2b952705c733 WatchSource:0}: Error finding container 51d09befa06958f5176aacfae3f4fd107a9b0a3b577ce2436f8b2b952705c733: Status 404 returned error can't find the container with id 51d09befa06958f5176aacfae3f4fd107a9b0a3b577ce2436f8b2b952705c733 Nov 04 05:38:06.384868 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:06.384833 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" event={"ID":"36073709-5d75-4ba9-9adf-fb35c540843b","Type":"ContainerStarted","Data":"51d09befa06958f5176aacfae3f4fd107a9b0a3b577ce2436f8b2b952705c733"} Nov 04 05:38:12.228926 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.228885 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-9m4gc"] Nov 04 05:38:12.241148 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.241117 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-9m4gc" Nov 04 05:38:12.245173 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.245120 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-9m4gc"] Nov 04 05:38:12.246751 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.246708 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Nov 04 05:38:12.246872 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.246765 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-8fd2h\"" Nov 04 05:38:12.253856 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.253347 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t"] Nov 04 05:38:12.257505 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.257488 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" Nov 04 05:38:12.266137 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.266104 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t"] Nov 04 05:38:12.280391 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.280367 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Nov 04 05:38:12.280733 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.280705 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-7zmm7\"" Nov 04 05:38:12.367752 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.367709 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp9jt\" (UniqueName: \"kubernetes.io/projected/e7305480-4dec-4da7-a70a-dd768b9901fe-kube-api-access-rp9jt\") pod \"llmisvc-controller-manager-869d5dcc5d-ssp9t\" (UID: \"e7305480-4dec-4da7-a70a-dd768b9901fe\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" Nov 04 05:38:12.367908 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.367839 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95e8955f-5e49-4bfb-8511-d2a57effca60-cert\") pod \"kserve-controller-manager-776cc66d47-9m4gc\" (UID: \"95e8955f-5e49-4bfb-8511-d2a57effca60\") " pod="opendatahub/kserve-controller-manager-776cc66d47-9m4gc" Nov 04 05:38:12.367955 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.367904 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jmjs\" (UniqueName: \"kubernetes.io/projected/95e8955f-5e49-4bfb-8511-d2a57effca60-kube-api-access-4jmjs\") pod \"kserve-controller-manager-776cc66d47-9m4gc\" (UID: \"95e8955f-5e49-4bfb-8511-d2a57effca60\") " pod="opendatahub/kserve-controller-manager-776cc66d47-9m4gc" Nov 04 05:38:12.367992 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.367960 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7305480-4dec-4da7-a70a-dd768b9901fe-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-ssp9t\" (UID: \"e7305480-4dec-4da7-a70a-dd768b9901fe\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" Nov 04 05:38:12.469278 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.469213 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jmjs\" (UniqueName: \"kubernetes.io/projected/95e8955f-5e49-4bfb-8511-d2a57effca60-kube-api-access-4jmjs\") pod \"kserve-controller-manager-776cc66d47-9m4gc\" (UID: \"95e8955f-5e49-4bfb-8511-d2a57effca60\") " pod="opendatahub/kserve-controller-manager-776cc66d47-9m4gc" Nov 04 05:38:12.469278 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.469281 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7305480-4dec-4da7-a70a-dd768b9901fe-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-ssp9t\" (UID: \"e7305480-4dec-4da7-a70a-dd768b9901fe\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" Nov 04 05:38:12.469536 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.469371 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp9jt\" (UniqueName: \"kubernetes.io/projected/e7305480-4dec-4da7-a70a-dd768b9901fe-kube-api-access-rp9jt\") pod \"llmisvc-controller-manager-869d5dcc5d-ssp9t\" (UID: \"e7305480-4dec-4da7-a70a-dd768b9901fe\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" Nov 04 05:38:12.469536 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.469443 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95e8955f-5e49-4bfb-8511-d2a57effca60-cert\") pod \"kserve-controller-manager-776cc66d47-9m4gc\" (UID: \"95e8955f-5e49-4bfb-8511-d2a57effca60\") " pod="opendatahub/kserve-controller-manager-776cc66d47-9m4gc" Nov 04 05:38:12.472362 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.472332 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95e8955f-5e49-4bfb-8511-d2a57effca60-cert\") pod \"kserve-controller-manager-776cc66d47-9m4gc\" (UID: \"95e8955f-5e49-4bfb-8511-d2a57effca60\") " pod="opendatahub/kserve-controller-manager-776cc66d47-9m4gc" Nov 04 05:38:12.472626 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.472601 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7305480-4dec-4da7-a70a-dd768b9901fe-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-ssp9t\" (UID: \"e7305480-4dec-4da7-a70a-dd768b9901fe\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" Nov 04 05:38:12.478817 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.478766 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp9jt\" (UniqueName: \"kubernetes.io/projected/e7305480-4dec-4da7-a70a-dd768b9901fe-kube-api-access-rp9jt\") pod \"llmisvc-controller-manager-869d5dcc5d-ssp9t\" (UID: \"e7305480-4dec-4da7-a70a-dd768b9901fe\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" Nov 04 05:38:12.479603 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.479549 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jmjs\" (UniqueName: \"kubernetes.io/projected/95e8955f-5e49-4bfb-8511-d2a57effca60-kube-api-access-4jmjs\") pod \"kserve-controller-manager-776cc66d47-9m4gc\" (UID: \"95e8955f-5e49-4bfb-8511-d2a57effca60\") " pod="opendatahub/kserve-controller-manager-776cc66d47-9m4gc" Nov 04 05:38:12.562954 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.562920 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-9m4gc" Nov 04 05:38:12.579972 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.579946 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" Nov 04 05:38:12.856582 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.856549 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-9m4gc"] Nov 04 05:38:12.858812 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:38:12.858769 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95e8955f_5e49_4bfb_8511_d2a57effca60.slice/crio-99128064dda38f25aac0ec6c4120d9a1de63f50fbf672937350ca8d2d3542f32 WatchSource:0}: Error finding container 99128064dda38f25aac0ec6c4120d9a1de63f50fbf672937350ca8d2d3542f32: Status 404 returned error can't find the container with id 99128064dda38f25aac0ec6c4120d9a1de63f50fbf672937350ca8d2d3542f32 Nov 04 05:38:12.890316 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:12.890286 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t"] Nov 04 05:38:12.894491 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:38:12.894460 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode7305480_4dec_4da7_a70a_dd768b9901fe.slice/crio-ebdd220fb53f42aa1bb2adf0030eb6b6ef1436c7db3234b49968589b86dd7b43 WatchSource:0}: Error finding container ebdd220fb53f42aa1bb2adf0030eb6b6ef1436c7db3234b49968589b86dd7b43: Status 404 returned error can't find the container with id ebdd220fb53f42aa1bb2adf0030eb6b6ef1436c7db3234b49968589b86dd7b43 Nov 04 05:38:13.434727 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:13.434625 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" event={"ID":"e7305480-4dec-4da7-a70a-dd768b9901fe","Type":"ContainerStarted","Data":"ebdd220fb53f42aa1bb2adf0030eb6b6ef1436c7db3234b49968589b86dd7b43"} Nov 04 05:38:13.436381 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:13.436345 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-9m4gc" event={"ID":"95e8955f-5e49-4bfb-8511-d2a57effca60","Type":"ContainerStarted","Data":"99128064dda38f25aac0ec6c4120d9a1de63f50fbf672937350ca8d2d3542f32"} Nov 04 05:38:15.381923 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:15.381884 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-595689b685-gmspv" Nov 04 05:38:16.461770 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:16.461730 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-9m4gc" event={"ID":"95e8955f-5e49-4bfb-8511-d2a57effca60","Type":"ContainerStarted","Data":"f1bb5304f0a424088d7b49774a35d6af32204d17e723e8ff6356a0b8f5cded94"} Nov 04 05:38:16.462288 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:16.461847 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-9m4gc" Nov 04 05:38:16.464001 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:16.463976 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" event={"ID":"e7305480-4dec-4da7-a70a-dd768b9901fe","Type":"ContainerStarted","Data":"4e756981aafa0549dd828f655a0c839956652c7275e1d88be91592e149d1a90b"} Nov 04 05:38:16.464136 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:16.464120 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" Nov 04 05:38:16.493072 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:16.493018 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-9m4gc" podStartSLOduration=1.7675382960000001 podStartE2EDuration="4.493003682s" totalImagesPullingTime="2.725465386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:16.477985286 +0000 UTC m=+543.435570236" watchObservedRunningTime="2026-11-04 05:38:16.493003682 +0000 UTC m=+543.450588632" Nov 04 05:38:16.493264 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:16.493118 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" podStartSLOduration=1.739476581 podStartE2EDuration="4.493111835s" totalImagesPullingTime="2.753635254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:16.492645042 +0000 UTC m=+543.450229991" watchObservedRunningTime="2026-11-04 05:38:16.493111835 +0000 UTC m=+543.450696786" Nov 04 05:38:18.476627 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:18.476531 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" event={"ID":"36073709-5d75-4ba9-9adf-fb35c540843b","Type":"ContainerStarted","Data":"0da5b8fbbe9097c8be1c512816ed18cda8c8ef6145b584e08107f5ab2c013ae2"} Nov 04 05:38:18.477056 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:18.476698 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" Nov 04 05:38:18.493916 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:18.493868 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" podStartSLOduration=1.858662834 podStartE2EDuration="14.493853885s" totalImagesPullingTime="12.635191051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:18.491905743 +0000 UTC m=+545.449490691" watchObservedRunningTime="2026-11-04 05:38:18.493853885 +0000 UTC m=+545.451438834" Nov 04 05:38:27.134409 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:38:27.134306 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:38:29.487381 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:29.487350 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-mkgsm" Nov 04 05:38:33.190318 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.190277 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq"] Nov 04 05:38:33.194066 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.194049 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:33.202138 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.202106 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Nov 04 05:38:33.202278 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.202214 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Nov 04 05:38:33.202340 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.202318 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dcrf5\"" Nov 04 05:38:33.204289 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.204260 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq"] Nov 04 05:38:33.256369 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.256333 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-openshift-service-ca\") pod \"maas-api-674d648f7b-7s9tq\" (UID: \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:33.256551 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.256377 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sphq4\" (UniqueName: \"kubernetes.io/projected/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-kube-api-access-sphq4\") pod \"maas-api-674d648f7b-7s9tq\" (UID: \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:33.256551 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.256492 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-maas-api-tls\") pod \"maas-api-674d648f7b-7s9tq\" (UID: \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:33.256661 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.256612 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-metrics-certs\") pod \"maas-api-674d648f7b-7s9tq\" (UID: \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:33.357838 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.357806 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-openshift-service-ca\") pod \"maas-api-674d648f7b-7s9tq\" (UID: \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:33.358053 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.357852 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sphq4\" (UniqueName: \"kubernetes.io/projected/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-kube-api-access-sphq4\") pod \"maas-api-674d648f7b-7s9tq\" (UID: \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:33.358053 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.357907 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-maas-api-tls\") pod \"maas-api-674d648f7b-7s9tq\" (UID: \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:33.358053 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.357997 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-metrics-certs\") pod \"maas-api-674d648f7b-7s9tq\" (UID: \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:33.358672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.358641 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-openshift-service-ca\") pod \"maas-api-674d648f7b-7s9tq\" (UID: \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:33.360577 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.360555 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-maas-api-tls\") pod \"maas-api-674d648f7b-7s9tq\" (UID: \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:33.360682 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.360621 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-metrics-certs\") pod \"maas-api-674d648f7b-7s9tq\" (UID: \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:33.365789 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.365762 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sphq4\" (UniqueName: \"kubernetes.io/projected/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-kube-api-access-sphq4\") pod \"maas-api-674d648f7b-7s9tq\" (UID: \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:33.509045 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.508940 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:33.784176 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:33.784113 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq"] Nov 04 05:38:33.786939 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:38:33.786905 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec04f573_7fd9_4bd9_bee8_561e3ddc5952.slice/crio-3b70276c15d7547599788d52218bd7ddb114676730d53703023c28ef7bb6c588 WatchSource:0}: Error finding container 3b70276c15d7547599788d52218bd7ddb114676730d53703023c28ef7bb6c588: Status 404 returned error can't find the container with id 3b70276c15d7547599788d52218bd7ddb114676730d53703023c28ef7bb6c588 Nov 04 05:38:34.555439 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:34.555400 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" event={"ID":"ec04f573-7fd9-4bd9-bee8-561e3ddc5952","Type":"ContainerStarted","Data":"3b70276c15d7547599788d52218bd7ddb114676730d53703023c28ef7bb6c588"} Nov 04 05:38:35.561488 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:35.561450 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" event={"ID":"ec04f573-7fd9-4bd9-bee8-561e3ddc5952","Type":"ContainerStarted","Data":"af1ed481fd1533eb84908ccbf8cb4c744cd900585f873fea1a08f5c745651d7b"} Nov 04 05:38:35.561955 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:35.561514 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:35.578032 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:35.577941 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" podStartSLOduration=1.052243111 podStartE2EDuration="2.577926485s" totalImagesPullingTime="1.525683374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:35.575791064 +0000 UTC m=+562.533376015" watchObservedRunningTime="2026-11-04 05:38:35.577926485 +0000 UTC m=+562.535511432" Nov 04 05:38:41.572118 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:41.572089 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:42.494427 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:42.494392 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp"] Nov 04 05:38:42.597730 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:42.597704 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp"] Nov 04 05:38:42.598140 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:42.597796 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" Nov 04 05:38:42.744355 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:42.744308 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8ebad6f-24c6-4160-b09b-4be3c3bec639-metrics-certs\") pod \"maas-api-5d8dd6bcb9-tbdvp\" (UID: \"c8ebad6f-24c6-4160-b09b-4be3c3bec639\") " pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" Nov 04 05:38:42.744578 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:42.744360 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ebad6f-24c6-4160-b09b-4be3c3bec639-openshift-service-ca\") pod \"maas-api-5d8dd6bcb9-tbdvp\" (UID: \"c8ebad6f-24c6-4160-b09b-4be3c3bec639\") " pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" Nov 04 05:38:42.744578 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:42.744494 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8ebad6f-24c6-4160-b09b-4be3c3bec639-maas-api-tls\") pod \"maas-api-5d8dd6bcb9-tbdvp\" (UID: \"c8ebad6f-24c6-4160-b09b-4be3c3bec639\") " pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" Nov 04 05:38:42.744578 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:42.744527 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pqch\" (UniqueName: \"kubernetes.io/projected/c8ebad6f-24c6-4160-b09b-4be3c3bec639-kube-api-access-7pqch\") pod \"maas-api-5d8dd6bcb9-tbdvp\" (UID: \"c8ebad6f-24c6-4160-b09b-4be3c3bec639\") " pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" Nov 04 05:38:42.845592 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:42.845553 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8ebad6f-24c6-4160-b09b-4be3c3bec639-metrics-certs\") pod \"maas-api-5d8dd6bcb9-tbdvp\" (UID: \"c8ebad6f-24c6-4160-b09b-4be3c3bec639\") " pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" Nov 04 05:38:42.845592 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:42.845596 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ebad6f-24c6-4160-b09b-4be3c3bec639-openshift-service-ca\") pod \"maas-api-5d8dd6bcb9-tbdvp\" (UID: \"c8ebad6f-24c6-4160-b09b-4be3c3bec639\") " pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" Nov 04 05:38:42.845843 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:42.845640 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8ebad6f-24c6-4160-b09b-4be3c3bec639-maas-api-tls\") pod \"maas-api-5d8dd6bcb9-tbdvp\" (UID: \"c8ebad6f-24c6-4160-b09b-4be3c3bec639\") " pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" Nov 04 05:38:42.845843 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:42.845660 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pqch\" (UniqueName: \"kubernetes.io/projected/c8ebad6f-24c6-4160-b09b-4be3c3bec639-kube-api-access-7pqch\") pod \"maas-api-5d8dd6bcb9-tbdvp\" (UID: \"c8ebad6f-24c6-4160-b09b-4be3c3bec639\") " pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" Nov 04 05:38:42.846406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:42.846382 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ebad6f-24c6-4160-b09b-4be3c3bec639-openshift-service-ca\") pod \"maas-api-5d8dd6bcb9-tbdvp\" (UID: \"c8ebad6f-24c6-4160-b09b-4be3c3bec639\") " pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" Nov 04 05:38:42.848157 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:42.848126 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8ebad6f-24c6-4160-b09b-4be3c3bec639-maas-api-tls\") pod \"maas-api-5d8dd6bcb9-tbdvp\" (UID: \"c8ebad6f-24c6-4160-b09b-4be3c3bec639\") " pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" Nov 04 05:38:42.848263 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:42.848219 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8ebad6f-24c6-4160-b09b-4be3c3bec639-metrics-certs\") pod \"maas-api-5d8dd6bcb9-tbdvp\" (UID: \"c8ebad6f-24c6-4160-b09b-4be3c3bec639\") " pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" Nov 04 05:38:42.852964 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:42.852941 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pqch\" (UniqueName: \"kubernetes.io/projected/c8ebad6f-24c6-4160-b09b-4be3c3bec639-kube-api-access-7pqch\") pod \"maas-api-5d8dd6bcb9-tbdvp\" (UID: \"c8ebad6f-24c6-4160-b09b-4be3c3bec639\") " pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" Nov 04 05:38:42.909944 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:42.909910 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" Nov 04 05:38:43.192340 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:43.192304 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp"] Nov 04 05:38:43.196306 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:38:43.196275 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8ebad6f_24c6_4160_b09b_4be3c3bec639.slice/crio-a58e1c79d5dfb8645c56ae0320f63b5476120ff00788d818be9d43b6100d4e59 WatchSource:0}: Error finding container a58e1c79d5dfb8645c56ae0320f63b5476120ff00788d818be9d43b6100d4e59: Status 404 returned error can't find the container with id a58e1c79d5dfb8645c56ae0320f63b5476120ff00788d818be9d43b6100d4e59 Nov 04 05:38:43.601051 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:43.601015 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" event={"ID":"c8ebad6f-24c6-4160-b09b-4be3c3bec639","Type":"ContainerStarted","Data":"a58e1c79d5dfb8645c56ae0320f63b5476120ff00788d818be9d43b6100d4e59"} Nov 04 05:38:44.607967 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:44.607931 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" event={"ID":"c8ebad6f-24c6-4160-b09b-4be3c3bec639","Type":"ContainerStarted","Data":"4a10f47b476cffd9625f65c9cee05218ad2d04467de842cf07acb56d2381aa81"} Nov 04 05:38:44.608368 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:44.608013 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" Nov 04 05:38:44.624758 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:44.624710 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" podStartSLOduration=2.220165239 podStartE2EDuration="2.624695535s" totalImagesPullingTime="404.530296ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:44.622879 +0000 UTC m=+571.580463965" watchObservedRunningTime="2026-11-04 05:38:44.624695535 +0000 UTC m=+571.582280824" Nov 04 05:38:47.471759 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:47.471719 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" Nov 04 05:38:47.474707 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:47.474687 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-9m4gc" Nov 04 05:38:50.618624 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:50.618594 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5d8dd6bcb9-tbdvp" Nov 04 05:38:50.659723 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:50.659687 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq"] Nov 04 05:38:50.659936 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:50.659909 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" podUID="ec04f573-7fd9-4bd9-bee8-561e3ddc5952" containerName="maas-api" containerID="cri-o://af1ed481fd1533eb84908ccbf8cb4c744cd900585f873fea1a08f5c745651d7b" gracePeriod=30 Nov 04 05:38:50.958855 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:50.958829 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:51.119071 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.119042 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-kube-api-access-sphq4\" (UniqueName: \"kubernetes.io/projected/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-kube-api-access-sphq4\") pod \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\" (UID: \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\") " Nov 04 05:38:51.119261 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.119079 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-metrics-certs\") pod \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\" (UID: \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\") " Nov 04 05:38:51.119261 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.119187 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-openshift-service-ca\") pod \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\" (UID: \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\") " Nov 04 05:38:51.119261 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.119251 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-maas-api-tls\") pod \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\" (UID: \"ec04f573-7fd9-4bd9-bee8-561e3ddc5952\") " Nov 04 05:38:51.119557 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.119532 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-openshift-service-ca" pod "ec04f573-7fd9-4bd9-bee8-561e3ddc5952" (UID: "ec04f573-7fd9-4bd9-bee8-561e3ddc5952"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:38:51.121549 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.121521 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-metrics-certs" pod "ec04f573-7fd9-4bd9-bee8-561e3ddc5952" (UID: "ec04f573-7fd9-4bd9-bee8-561e3ddc5952"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:38:51.121706 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.121686 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-maas-api-tls" pod "ec04f573-7fd9-4bd9-bee8-561e3ddc5952" (UID: "ec04f573-7fd9-4bd9-bee8-561e3ddc5952"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:38:51.121774 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.121716 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-kube-api-access-sphq4" pod "ec04f573-7fd9-4bd9-bee8-561e3ddc5952" (UID: "ec04f573-7fd9-4bd9-bee8-561e3ddc5952"). InnerVolumeSpecName "kube-api-access-sphq4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:38:51.220482 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.220449 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sphq4\" (UniqueName: \"kubernetes.io/projected/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-kube-api-access-sphq4\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:38:51.220482 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.220477 5904 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-metrics-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:38:51.220482 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.220488 5904 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-openshift-service-ca\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:38:51.220705 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.220497 5904 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec04f573-7fd9-4bd9-bee8-561e3ddc5952-maas-api-tls\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:38:51.648813 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.648777 5904 generic.go:356] "Generic (PLEG): container finished" podID="ec04f573-7fd9-4bd9-bee8-561e3ddc5952" containerID="af1ed481fd1533eb84908ccbf8cb4c744cd900585f873fea1a08f5c745651d7b" exitCode=0 Nov 04 05:38:51.649296 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.648826 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" Nov 04 05:38:51.649296 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.648856 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" event={"ID":"ec04f573-7fd9-4bd9-bee8-561e3ddc5952","Type":"ContainerDied","Data":"af1ed481fd1533eb84908ccbf8cb4c744cd900585f873fea1a08f5c745651d7b"} Nov 04 05:38:51.649296 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.648914 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq" event={"ID":"ec04f573-7fd9-4bd9-bee8-561e3ddc5952","Type":"ContainerDied","Data":"3b70276c15d7547599788d52218bd7ddb114676730d53703023c28ef7bb6c588"} Nov 04 05:38:51.649296 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.648940 5904 scope.go:122] "RemoveContainer" containerID="af1ed481fd1533eb84908ccbf8cb4c744cd900585f873fea1a08f5c745651d7b" Nov 04 05:38:51.657747 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.657726 5904 scope.go:122] "RemoveContainer" containerID="af1ed481fd1533eb84908ccbf8cb4c744cd900585f873fea1a08f5c745651d7b" Nov 04 05:38:51.658016 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:38:51.657997 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af1ed481fd1533eb84908ccbf8cb4c744cd900585f873fea1a08f5c745651d7b\": container with ID starting with af1ed481fd1533eb84908ccbf8cb4c744cd900585f873fea1a08f5c745651d7b not found: ID does not exist" containerID="af1ed481fd1533eb84908ccbf8cb4c744cd900585f873fea1a08f5c745651d7b" Nov 04 05:38:51.658073 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.658026 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af1ed481fd1533eb84908ccbf8cb4c744cd900585f873fea1a08f5c745651d7b"} err="failed to get container status \"af1ed481fd1533eb84908ccbf8cb4c744cd900585f873fea1a08f5c745651d7b\": rpc error: code = NotFound desc = could not find container \"af1ed481fd1533eb84908ccbf8cb4c744cd900585f873fea1a08f5c745651d7b\": container with ID starting with af1ed481fd1533eb84908ccbf8cb4c744cd900585f873fea1a08f5c745651d7b not found: ID does not exist" Nov 04 05:38:51.664365 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.664341 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq"] Nov 04 05:38:51.668174 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:51.668153 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-7s9tq"] Nov 04 05:38:53.167566 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:38:53.167534 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec04f573-7fd9-4bd9-bee8-561e3ddc5952" path="/var/lib/kubelet/pods/ec04f573-7fd9-4bd9-bee8-561e3ddc5952/volumes" Nov 04 05:38:57.131665 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:38:57.131619 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:39:06.328747 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:06.328715 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-l8cn6"] Nov 04 05:39:06.329286 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:06.329098 5904 cpu_manager.go:414] "containerMap: removing container" podUID="ec04f573-7fd9-4bd9-bee8-561e3ddc5952" containerName="maas-api" Nov 04 05:39:06.329286 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:06.329109 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec04f573-7fd9-4bd9-bee8-561e3ddc5952" containerName="maas-api" Nov 04 05:39:06.329286 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:06.329196 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec04f573-7fd9-4bd9-bee8-561e3ddc5952" containerName="maas-api" Nov 04 05:39:06.333253 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:06.333214 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-l8cn6" Nov 04 05:39:06.341144 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:06.341120 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-l8cn6"] Nov 04 05:39:06.354737 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:06.354704 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7sl5\" (UniqueName: \"kubernetes.io/projected/1f43099e-9c80-4bc5-9a7a-18d3e22534f9-kube-api-access-c7sl5\") pod \"llmisvc-controller-manager-6466c554d8-l8cn6\" (UID: \"1f43099e-9c80-4bc5-9a7a-18d3e22534f9\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-l8cn6" Nov 04 05:39:06.354883 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:06.354863 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f43099e-9c80-4bc5-9a7a-18d3e22534f9-cert\") pod \"llmisvc-controller-manager-6466c554d8-l8cn6\" (UID: \"1f43099e-9c80-4bc5-9a7a-18d3e22534f9\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-l8cn6" Nov 04 05:39:06.456275 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:06.456215 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f43099e-9c80-4bc5-9a7a-18d3e22534f9-cert\") pod \"llmisvc-controller-manager-6466c554d8-l8cn6\" (UID: \"1f43099e-9c80-4bc5-9a7a-18d3e22534f9\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-l8cn6" Nov 04 05:39:06.456442 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:06.456317 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7sl5\" (UniqueName: \"kubernetes.io/projected/1f43099e-9c80-4bc5-9a7a-18d3e22534f9-kube-api-access-c7sl5\") pod \"llmisvc-controller-manager-6466c554d8-l8cn6\" (UID: \"1f43099e-9c80-4bc5-9a7a-18d3e22534f9\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-l8cn6" Nov 04 05:39:06.458705 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:06.458681 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f43099e-9c80-4bc5-9a7a-18d3e22534f9-cert\") pod \"llmisvc-controller-manager-6466c554d8-l8cn6\" (UID: \"1f43099e-9c80-4bc5-9a7a-18d3e22534f9\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-l8cn6" Nov 04 05:39:06.463953 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:06.463927 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7sl5\" (UniqueName: \"kubernetes.io/projected/1f43099e-9c80-4bc5-9a7a-18d3e22534f9-kube-api-access-c7sl5\") pod \"llmisvc-controller-manager-6466c554d8-l8cn6\" (UID: \"1f43099e-9c80-4bc5-9a7a-18d3e22534f9\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-l8cn6" Nov 04 05:39:06.648400 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:06.648316 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-l8cn6" Nov 04 05:39:06.919303 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:06.919279 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-l8cn6"] Nov 04 05:39:06.921842 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:39:06.921814 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f43099e_9c80_4bc5_9a7a_18d3e22534f9.slice/crio-6deed62a0547457917e5312addfa245891451474181cd0cb518754d2966e26fd WatchSource:0}: Error finding container 6deed62a0547457917e5312addfa245891451474181cd0cb518754d2966e26fd: Status 404 returned error can't find the container with id 6deed62a0547457917e5312addfa245891451474181cd0cb518754d2966e26fd Nov 04 05:39:07.733487 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:07.733405 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-l8cn6" event={"ID":"1f43099e-9c80-4bc5-9a7a-18d3e22534f9","Type":"ContainerStarted","Data":"73936f0609f8df1549c017b1c6fde17eae9f7e97f69888a18cb5b21acbb7004b"} Nov 04 05:39:07.733487 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:07.733456 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-l8cn6" event={"ID":"1f43099e-9c80-4bc5-9a7a-18d3e22534f9","Type":"ContainerStarted","Data":"6deed62a0547457917e5312addfa245891451474181cd0cb518754d2966e26fd"} Nov 04 05:39:07.733487 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:07.733489 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-l8cn6" Nov 04 05:39:07.749670 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:07.749622 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-l8cn6" podStartSLOduration=1.347246891 podStartE2EDuration="1.74961s" totalImagesPullingTime="402.363109ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:39:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:39:07.747995432 +0000 UTC m=+594.705580381" watchObservedRunningTime="2026-11-04 05:39:07.74961 +0000 UTC m=+594.707194949" Nov 04 05:39:13.239895 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:13.239870 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-slrx2_82660df8-f278-4d7c-b02a-149527e30d27/ovn-acl-logging/0.log" Nov 04 05:39:13.243361 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:13.243339 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-slrx2_82660df8-f278-4d7c-b02a-149527e30d27/ovn-acl-logging/0.log" Nov 04 05:39:27.131372 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:39:27.131329 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:39:38.741399 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:38.741369 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-l8cn6" Nov 04 05:39:38.783645 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:38.783604 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t"] Nov 04 05:39:38.783900 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:38.783846 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" podUID="e7305480-4dec-4da7-a70a-dd768b9901fe" containerName="manager" containerID="cri-o://4e756981aafa0549dd828f655a0c839956652c7275e1d88be91592e149d1a90b" gracePeriod=30 Nov 04 05:39:39.027204 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:39.027175 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" Nov 04 05:39:39.159867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:39.159818 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e7305480-4dec-4da7-a70a-dd768b9901fe-cert\" (UniqueName: \"kubernetes.io/secret/e7305480-4dec-4da7-a70a-dd768b9901fe-cert\") pod \"e7305480-4dec-4da7-a70a-dd768b9901fe\" (UID: \"e7305480-4dec-4da7-a70a-dd768b9901fe\") " Nov 04 05:39:39.160047 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:39.159947 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7305480-4dec-4da7-a70a-dd768b9901fe-kube-api-access-rp9jt\" (UniqueName: \"kubernetes.io/projected/e7305480-4dec-4da7-a70a-dd768b9901fe-kube-api-access-rp9jt\") pod \"e7305480-4dec-4da7-a70a-dd768b9901fe\" (UID: \"e7305480-4dec-4da7-a70a-dd768b9901fe\") " Nov 04 05:39:39.162070 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:39.162040 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7305480-4dec-4da7-a70a-dd768b9901fe-cert" pod "e7305480-4dec-4da7-a70a-dd768b9901fe" (UID: "e7305480-4dec-4da7-a70a-dd768b9901fe"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:39:39.162206 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:39.162166 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7305480-4dec-4da7-a70a-dd768b9901fe-kube-api-access-rp9jt" pod "e7305480-4dec-4da7-a70a-dd768b9901fe" (UID: "e7305480-4dec-4da7-a70a-dd768b9901fe"). InnerVolumeSpecName "kube-api-access-rp9jt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:39:39.261361 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:39.261289 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rp9jt\" (UniqueName: \"kubernetes.io/projected/e7305480-4dec-4da7-a70a-dd768b9901fe-kube-api-access-rp9jt\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:39:39.261361 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:39.261322 5904 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7305480-4dec-4da7-a70a-dd768b9901fe-cert\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:39:39.893341 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:39.893307 5904 generic.go:356] "Generic (PLEG): container finished" podID="e7305480-4dec-4da7-a70a-dd768b9901fe" containerID="4e756981aafa0549dd828f655a0c839956652c7275e1d88be91592e149d1a90b" exitCode=0 Nov 04 05:39:39.893788 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:39.893353 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" Nov 04 05:39:39.893788 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:39.893391 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" event={"ID":"e7305480-4dec-4da7-a70a-dd768b9901fe","Type":"ContainerDied","Data":"4e756981aafa0549dd828f655a0c839956652c7275e1d88be91592e149d1a90b"} Nov 04 05:39:39.893788 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:39.893451 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t" event={"ID":"e7305480-4dec-4da7-a70a-dd768b9901fe","Type":"ContainerDied","Data":"ebdd220fb53f42aa1bb2adf0030eb6b6ef1436c7db3234b49968589b86dd7b43"} Nov 04 05:39:39.893788 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:39.893478 5904 scope.go:122] "RemoveContainer" containerID="4e756981aafa0549dd828f655a0c839956652c7275e1d88be91592e149d1a90b" Nov 04 05:39:39.904585 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:39.904558 5904 scope.go:122] "RemoveContainer" containerID="4e756981aafa0549dd828f655a0c839956652c7275e1d88be91592e149d1a90b" Nov 04 05:39:39.904966 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:39:39.904945 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e756981aafa0549dd828f655a0c839956652c7275e1d88be91592e149d1a90b\": container with ID starting with 4e756981aafa0549dd828f655a0c839956652c7275e1d88be91592e149d1a90b not found: ID does not exist" containerID="4e756981aafa0549dd828f655a0c839956652c7275e1d88be91592e149d1a90b" Nov 04 05:39:39.905033 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:39.904978 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e756981aafa0549dd828f655a0c839956652c7275e1d88be91592e149d1a90b"} err="failed to get container status \"4e756981aafa0549dd828f655a0c839956652c7275e1d88be91592e149d1a90b\": rpc error: code = NotFound desc = could not find container \"4e756981aafa0549dd828f655a0c839956652c7275e1d88be91592e149d1a90b\": container with ID starting with 4e756981aafa0549dd828f655a0c839956652c7275e1d88be91592e149d1a90b not found: ID does not exist" Nov 04 05:39:39.909395 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:39.909371 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t"] Nov 04 05:39:39.913081 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:39.913060 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ssp9t"] Nov 04 05:39:41.167367 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:39:41.167335 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7305480-4dec-4da7-a70a-dd768b9901fe" path="/var/lib/kubelet/pods/e7305480-4dec-4da7-a70a-dd768b9901fe/volumes" Nov 04 05:39:57.132537 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:39:57.132456 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:40:27.133853 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:40:27.133813 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:40:31.146111 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:31.146076 5904 generic.go:356] "Generic (PLEG): container finished" podID="57c93381-b98d-4d00-baf7-1b31d0f6c433" containerID="06b1c6823e4ce479278c2fe437af528f6e24f81be77acfe927bb4a12ba2a69a8" exitCode=1 Nov 04 05:40:31.146111 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:31.146114 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-595689b685-gmspv" event={"ID":"57c93381-b98d-4d00-baf7-1b31d0f6c433","Type":"ContainerDied","Data":"06b1c6823e4ce479278c2fe437af528f6e24f81be77acfe927bb4a12ba2a69a8"} Nov 04 05:40:31.146547 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:31.146488 5904 scope.go:122] "RemoveContainer" containerID="06b1c6823e4ce479278c2fe437af528f6e24f81be77acfe927bb4a12ba2a69a8" Nov 04 05:40:32.154441 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:32.154409 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-595689b685-gmspv" event={"ID":"57c93381-b98d-4d00-baf7-1b31d0f6c433","Type":"ContainerStarted","Data":"96d7c287c7d3c91bc72d064738f87889aab784737d04c8fddd3dabe82f4d84a9"} Nov 04 05:40:32.154849 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:32.154639 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-595689b685-gmspv" Nov 04 05:40:38.633381 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.633345 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s"] Nov 04 05:40:38.633768 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.633759 5904 cpu_manager.go:414] "containerMap: removing container" podUID="e7305480-4dec-4da7-a70a-dd768b9901fe" containerName="manager" Nov 04 05:40:38.633809 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.633770 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7305480-4dec-4da7-a70a-dd768b9901fe" containerName="manager" Nov 04 05:40:38.633856 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.633846 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7305480-4dec-4da7-a70a-dd768b9901fe" containerName="manager" Nov 04 05:40:38.636896 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.636876 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.640263 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.640241 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Nov 04 05:40:38.640412 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.640397 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-j9xrs\"" Nov 04 05:40:38.650302 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.650279 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s"] Nov 04 05:40:38.786098 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.786062 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87f3858d-ee78-4468-8b48-16073a80b645-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.786313 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.786116 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87f3858d-ee78-4468-8b48-16073a80b645-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.786313 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.786264 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tkbh\" (UniqueName: \"kubernetes.io/projected/87f3858d-ee78-4468-8b48-16073a80b645-kube-api-access-6tkbh\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.786313 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.786301 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87f3858d-ee78-4468-8b48-16073a80b645-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.786440 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.786325 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87f3858d-ee78-4468-8b48-16073a80b645-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.786440 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.786411 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87f3858d-ee78-4468-8b48-16073a80b645-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.786578 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.786439 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87f3858d-ee78-4468-8b48-16073a80b645-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.786578 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.786468 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87f3858d-ee78-4468-8b48-16073a80b645-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.786680 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.786577 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87f3858d-ee78-4468-8b48-16073a80b645-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.888124 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.888041 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87f3858d-ee78-4468-8b48-16073a80b645-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.888124 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.888077 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87f3858d-ee78-4468-8b48-16073a80b645-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.888124 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.888114 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87f3858d-ee78-4468-8b48-16073a80b645-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.888393 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.888173 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87f3858d-ee78-4468-8b48-16073a80b645-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.888393 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.888247 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tkbh\" (UniqueName: \"kubernetes.io/projected/87f3858d-ee78-4468-8b48-16073a80b645-kube-api-access-6tkbh\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.888393 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.888286 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87f3858d-ee78-4468-8b48-16073a80b645-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.888393 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.888318 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87f3858d-ee78-4468-8b48-16073a80b645-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.888393 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.888373 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87f3858d-ee78-4468-8b48-16073a80b645-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.888665 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.888397 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87f3858d-ee78-4468-8b48-16073a80b645-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.888665 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.888477 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87f3858d-ee78-4468-8b48-16073a80b645-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.888665 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.888553 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87f3858d-ee78-4468-8b48-16073a80b645-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.888799 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.888692 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87f3858d-ee78-4468-8b48-16073a80b645-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.888840 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.888826 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87f3858d-ee78-4468-8b48-16073a80b645-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.889096 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.889075 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87f3858d-ee78-4468-8b48-16073a80b645-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.890706 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.890679 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87f3858d-ee78-4468-8b48-16073a80b645-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.890799 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.890704 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87f3858d-ee78-4468-8b48-16073a80b645-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.895538 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.895518 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87f3858d-ee78-4468-8b48-16073a80b645-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.895644 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.895604 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tkbh\" (UniqueName: \"kubernetes.io/projected/87f3858d-ee78-4468-8b48-16073a80b645-kube-api-access-6tkbh\") pod \"maas-default-gateway-openshift-default-694db55c87-7rc4s\" (UID: \"87f3858d-ee78-4468-8b48-16073a80b645\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:38.950627 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:38.950598 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:39.226895 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:39.226868 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s"] Nov 04 05:40:39.229036 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:40:39.229008 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87f3858d_ee78_4468_8b48_16073a80b645.slice/crio-c0bc0022e18607486ece0cb67c91ea3980feab3d0972d357dc165cbd6b907f3e WatchSource:0}: Error finding container c0bc0022e18607486ece0cb67c91ea3980feab3d0972d357dc165cbd6b907f3e: Status 404 returned error can't find the container with id c0bc0022e18607486ece0cb67c91ea3980feab3d0972d357dc165cbd6b907f3e Nov 04 05:40:40.192483 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:40.192444 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" event={"ID":"87f3858d-ee78-4468-8b48-16073a80b645","Type":"ContainerStarted","Data":"c0bc0022e18607486ece0cb67c91ea3980feab3d0972d357dc165cbd6b907f3e"} Nov 04 05:40:42.291781 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:42.291744 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:40:42.292181 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:42.291828 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:40:42.292181 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:42.291872 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:40:43.176661 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:43.176632 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-595689b685-gmspv" Nov 04 05:40:43.208421 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:43.208387 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" event={"ID":"87f3858d-ee78-4468-8b48-16073a80b645","Type":"ContainerStarted","Data":"ef93568af2d415fc9cdda14153c2ed682a99b4820f331d189fb16d514699a392"} Nov 04 05:40:43.229081 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:43.229021 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" podStartSLOduration=2.169350302 podStartE2EDuration="5.229000822s" totalImagesPullingTime="3.05965052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:40:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:40:43.226405576 +0000 UTC m=+690.183990537" watchObservedRunningTime="2026-11-04 05:40:43.229000822 +0000 UTC m=+690.186585776" Nov 04 05:40:43.959250 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:43.956753 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:43.959250 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:43.956907 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-7rc4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:40:43.959250 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:43.956954 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" podUID="87f3858d-ee78-4468-8b48-16073a80b645" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Nov 04 05:40:44.959710 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:44.959674 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-7rc4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:40:44.960340 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:44.960305 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" podUID="87f3858d-ee78-4468-8b48-16073a80b645" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Nov 04 05:40:45.957251 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:45.952636 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-7rc4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:40:45.957251 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:45.952714 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" podUID="87f3858d-ee78-4468-8b48-16073a80b645" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Nov 04 05:40:46.953252 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:46.952617 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-7rc4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:40:46.953252 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:46.952681 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" podUID="87f3858d-ee78-4468-8b48-16073a80b645" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Nov 04 05:40:47.958649 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:47.958609 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-7rc4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:40:47.959306 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:47.959248 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" podUID="87f3858d-ee78-4468-8b48-16073a80b645" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Nov 04 05:40:48.951185 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:48.951150 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-7rc4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:40:48.951383 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:48.951204 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" podUID="87f3858d-ee78-4468-8b48-16073a80b645" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Nov 04 05:40:50.874556 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:50.874520 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:50.875172 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:50.875143 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:50.875976 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:50.875953 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-7rc4s" Nov 04 05:40:53.672362 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.672317 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9"] Nov 04 05:40:53.677391 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.677364 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.691762 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.691731 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9"] Nov 04 05:40:53.755046 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.755009 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.755276 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.755073 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.755276 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.755201 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.755428 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.755306 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhp8w\" (UniqueName: \"kubernetes.io/projected/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-kube-api-access-vhp8w\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.755428 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.755368 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.755428 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.755411 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.755603 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.755483 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.755603 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.755513 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.755603 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.755541 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.856457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.856414 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhp8w\" (UniqueName: \"kubernetes.io/projected/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-kube-api-access-vhp8w\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.856457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.856469 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.856737 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.856511 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.856737 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.856602 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.856737 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.856650 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.856737 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.856681 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.856985 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.856786 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.856985 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.856831 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.856985 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.856865 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.857164 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.857068 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.857370 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.857284 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.857575 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.857551 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.857662 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.857634 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.858052 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.858030 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.859903 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.859880 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.859903 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.859891 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.864584 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.864559 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:53.864706 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:53.864627 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhp8w\" (UniqueName: \"kubernetes.io/projected/c671ccc5-92a6-4b7d-8a92-7955bd3e15fd-kube-api-access-vhp8w\") pod \"maas-default-gateway-openshift-default-694db55c87-zfft9\" (UID: \"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:54.000450 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:54.000351 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:54.301616 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:54.301064 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9"] Nov 04 05:40:54.303338 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:40:54.303291 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc671ccc5_92a6_4b7d_8a92_7955bd3e15fd.slice/crio-609fd4a2484d0d4082250fd0689150c39edf1713aba6b727e2be08b205a07e6b WatchSource:0}: Error finding container 609fd4a2484d0d4082250fd0689150c39edf1713aba6b727e2be08b205a07e6b: Status 404 returned error can't find the container with id 609fd4a2484d0d4082250fd0689150c39edf1713aba6b727e2be08b205a07e6b Nov 04 05:40:54.306316 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:54.306283 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:40:54.306442 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:54.306353 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:40:54.306442 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:54.306383 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:40:55.276910 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:55.276877 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" event={"ID":"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd","Type":"ContainerStarted","Data":"453154968b3527378e8795c770065071a3333734b67782ae67e0df424a126d68"} Nov 04 05:40:55.277330 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:55.276930 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" event={"ID":"c671ccc5-92a6-4b7d-8a92-7955bd3e15fd","Type":"ContainerStarted","Data":"609fd4a2484d0d4082250fd0689150c39edf1713aba6b727e2be08b205a07e6b"} Nov 04 05:40:55.303724 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:55.303676 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" podStartSLOduration=2.303663356 podStartE2EDuration="2.303663356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:40:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:40:55.303557018 +0000 UTC m=+702.261141969" watchObservedRunningTime="2026-11-04 05:40:55.303663356 +0000 UTC m=+702.261248307" Nov 04 05:40:56.000571 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:56.000539 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:40:56.002050 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:56.002025 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-zfft9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:40:56.002185 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:56.002073 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" podUID="c671ccc5-92a6-4b7d-8a92-7955bd3e15fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Nov 04 05:40:57.000983 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:57.000944 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-zfft9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:40:57.001413 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:57.001009 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" podUID="c671ccc5-92a6-4b7d-8a92-7955bd3e15fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Nov 04 05:40:57.132348 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:40:57.132314 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:40:58.001024 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:58.000988 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-zfft9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:40:58.001465 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:58.001052 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" podUID="c671ccc5-92a6-4b7d-8a92-7955bd3e15fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Nov 04 05:40:59.000912 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:59.000875 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-zfft9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:40:59.001101 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:40:59.000931 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" podUID="c671ccc5-92a6-4b7d-8a92-7955bd3e15fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Nov 04 05:41:00.000918 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:00.000882 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-zfft9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:41:00.001103 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:00.000940 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" podUID="c671ccc5-92a6-4b7d-8a92-7955bd3e15fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Nov 04 05:41:01.001242 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:01.001196 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-zfft9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:41:01.001669 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:01.001262 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" podUID="c671ccc5-92a6-4b7d-8a92-7955bd3e15fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Nov 04 05:41:02.001430 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:02.001397 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-zfft9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:41:02.001869 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:02.001461 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" podUID="c671ccc5-92a6-4b7d-8a92-7955bd3e15fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Nov 04 05:41:04.042411 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:04.042372 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-zfft9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 05:41:04.042800 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:04.042467 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" podUID="c671ccc5-92a6-4b7d-8a92-7955bd3e15fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 05:41:05.084433 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:05.084387 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-zfft9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 05:41:05.084850 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:05.084479 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" podUID="c671ccc5-92a6-4b7d-8a92-7955bd3e15fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 05:41:05.331487 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:05.331455 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:41:05.332128 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:05.332093 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:41:05.332887 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:05.332868 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-zfft9" Nov 04 05:41:08.680715 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.680677 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4"] Nov 04 05:41:08.684516 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.684491 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.696856 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.696821 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4"] Nov 04 05:41:08.809237 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.809191 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.809426 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.809261 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.809426 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.809334 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.809426 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.809368 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.809426 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.809385 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.809595 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.809476 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj9tl\" (UniqueName: \"kubernetes.io/projected/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-kube-api-access-jj9tl\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.809595 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.809501 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.809656 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.809600 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.809656 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.809644 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.910977 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.910927 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.911101 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.910987 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.911101 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.911017 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.911101 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.911036 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.911101 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.911072 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj9tl\" (UniqueName: \"kubernetes.io/projected/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-kube-api-access-jj9tl\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.911336 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.911106 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.911336 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.911145 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.911336 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.911184 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.911336 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.911276 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.911623 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.911597 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.911701 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.911597 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.911701 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.911635 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.911812 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.911759 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.912138 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.912118 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.913486 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.913464 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.913583 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.913506 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.919605 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.919581 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.919846 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.919827 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj9tl\" (UniqueName: \"kubernetes.io/projected/8ca67e76-2ec7-4142-a8c3-0493d3a1cc63-kube-api-access-jj9tl\") pod \"maas-default-gateway-openshift-default-694db55c87-hwbs4\" (UID: \"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:08.999080 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:08.999001 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:09.275917 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:09.275893 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4"] Nov 04 05:41:09.278744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:41:09.278715 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ca67e76_2ec7_4142_a8c3_0493d3a1cc63.slice/crio-240b9de4d83c83ff15cc7be19d4a859a36f6b55908795ff4065fc26725bc8616 WatchSource:0}: Error finding container 240b9de4d83c83ff15cc7be19d4a859a36f6b55908795ff4065fc26725bc8616: Status 404 returned error can't find the container with id 240b9de4d83c83ff15cc7be19d4a859a36f6b55908795ff4065fc26725bc8616 Nov 04 05:41:09.281215 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:09.281175 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:41:09.281338 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:09.281297 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:41:09.281399 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:09.281353 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:41:09.349405 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:09.349370 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" event={"ID":"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63","Type":"ContainerStarted","Data":"a1a696c5defe8df09f8cfdd0018ee60d347b54f022e6d7e1cfddbdf9692986e6"} Nov 04 05:41:09.349527 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:09.349442 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" event={"ID":"8ca67e76-2ec7-4142-a8c3-0493d3a1cc63","Type":"ContainerStarted","Data":"240b9de4d83c83ff15cc7be19d4a859a36f6b55908795ff4065fc26725bc8616"} Nov 04 05:41:09.370052 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:09.370000 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" podStartSLOduration=1.3699861709999999 podStartE2EDuration="1.369986171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:41:09.367889035 +0000 UTC m=+716.325473985" watchObservedRunningTime="2026-11-04 05:41:09.369986171 +0000 UTC m=+716.327571190" Nov 04 05:41:09.999914 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:09.999880 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:11.000873 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:11.000839 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-hwbs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Nov 04 05:41:11.001271 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:11.000892 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" podUID="8ca67e76-2ec7-4142-a8c3-0493d3a1cc63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Nov 04 05:41:11.999516 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:11.999481 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-hwbs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Nov 04 05:41:11.999672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:11.999535 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" podUID="8ca67e76-2ec7-4142-a8c3-0493d3a1cc63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Nov 04 05:41:13.000185 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:13.000145 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-hwbs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Nov 04 05:41:13.000607 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:13.000212 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" podUID="8ca67e76-2ec7-4142-a8c3-0493d3a1cc63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Nov 04 05:41:13.999654 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:13.999617 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-hwbs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Nov 04 05:41:13.999809 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:13.999672 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" podUID="8ca67e76-2ec7-4142-a8c3-0493d3a1cc63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Nov 04 05:41:14.999862 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:14.999826 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-hwbs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Nov 04 05:41:15.000371 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:14.999886 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" podUID="8ca67e76-2ec7-4142-a8c3-0493d3a1cc63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Nov 04 05:41:16.000026 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:15.999992 5904 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-hwbs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Nov 04 05:41:16.000531 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:16.000050 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" podUID="8ca67e76-2ec7-4142-a8c3-0493d3a1cc63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Nov 04 05:41:17.826899 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:17.826858 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:17.827424 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:17.827254 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:17.828037 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:17.828017 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hwbs4" Nov 04 05:41:27.131684 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:41:27.131600 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:41:46.558292 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:46.558256 5904 generic.go:356] "Generic (PLEG): container finished" podID="1bc49b45-21c3-4c36-871b-157e538dea5f" containerID="db4fa60acfe959314be4db87f016335f5a7bfa06361d25da342f540695b1bf35" exitCode=1 Nov 04 05:41:46.558676 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:46.558315 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hktgm" event={"ID":"1bc49b45-21c3-4c36-871b-157e538dea5f","Type":"ContainerDied","Data":"db4fa60acfe959314be4db87f016335f5a7bfa06361d25da342f540695b1bf35"} Nov 04 05:41:46.558676 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:46.558627 5904 scope.go:122] "RemoveContainer" containerID="db4fa60acfe959314be4db87f016335f5a7bfa06361d25da342f540695b1bf35" Nov 04 05:41:47.565757 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:41:47.565718 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hktgm" event={"ID":"1bc49b45-21c3-4c36-871b-157e538dea5f","Type":"ContainerStarted","Data":"eb5dd9f121fda06c139145d2236aa11d3d9c28fd85e605035c98a2a7a6b66402"} Nov 04 05:41:57.131612 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:41:57.131569 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:42:26.798059 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:26.798022 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l"] Nov 04 05:42:26.811670 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:26.811219 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:42:26.811891 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:26.811122 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l"] Nov 04 05:42:26.878599 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:26.878565 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-f910f5-serving-cert\"" Nov 04 05:42:26.908467 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:26.908439 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b1fde1-970f-4864-ac83-a2ce6110cbad-maas-api-tls\") pod \"maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l\" (UID: \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:42:26.908628 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:26.908474 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8fpz\" (UniqueName: \"kubernetes.io/projected/a9b1fde1-970f-4864-ac83-a2ce6110cbad-kube-api-access-k8fpz\") pod \"maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l\" (UID: \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:42:26.908628 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:26.908610 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9b1fde1-970f-4864-ac83-a2ce6110cbad-metrics-certs\") pod \"maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l\" (UID: \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:42:26.908707 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:26.908646 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b1fde1-970f-4864-ac83-a2ce6110cbad-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l\" (UID: \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:42:27.009679 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:27.009643 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9b1fde1-970f-4864-ac83-a2ce6110cbad-metrics-certs\") pod \"maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l\" (UID: \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:42:27.009863 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:27.009683 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b1fde1-970f-4864-ac83-a2ce6110cbad-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l\" (UID: \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:42:27.009863 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:27.009747 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b1fde1-970f-4864-ac83-a2ce6110cbad-maas-api-tls\") pod \"maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l\" (UID: \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:42:27.009863 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:27.009780 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8fpz\" (UniqueName: \"kubernetes.io/projected/a9b1fde1-970f-4864-ac83-a2ce6110cbad-kube-api-access-k8fpz\") pod \"maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l\" (UID: \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:42:27.010487 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:27.010460 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b1fde1-970f-4864-ac83-a2ce6110cbad-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l\" (UID: \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:42:27.012179 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:27.012149 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9b1fde1-970f-4864-ac83-a2ce6110cbad-metrics-certs\") pod \"maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l\" (UID: \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:42:27.012314 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:27.012192 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b1fde1-970f-4864-ac83-a2ce6110cbad-maas-api-tls\") pod \"maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l\" (UID: \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:42:27.017199 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:27.017169 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8fpz\" (UniqueName: \"kubernetes.io/projected/a9b1fde1-970f-4864-ac83-a2ce6110cbad-kube-api-access-k8fpz\") pod \"maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l\" (UID: \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:42:27.130404 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:27.130374 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:42:27.132627 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:42:27.132599 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:42:27.420756 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:27.420661 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l"] Nov 04 05:42:27.424663 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:42:27.424633 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b1fde1_970f_4864_ac83_a2ce6110cbad.slice/crio-72f0d2442ee3ddbc9fec10f3ac259baaa5098f837d24b9bb19bbece9191ca6aa WatchSource:0}: Error finding container 72f0d2442ee3ddbc9fec10f3ac259baaa5098f837d24b9bb19bbece9191ca6aa: Status 404 returned error can't find the container with id 72f0d2442ee3ddbc9fec10f3ac259baaa5098f837d24b9bb19bbece9191ca6aa Nov 04 05:42:27.426014 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:27.425995 5904 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:42:27.763542 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:27.763444 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" event={"ID":"a9b1fde1-970f-4864-ac83-a2ce6110cbad","Type":"ContainerStarted","Data":"72f0d2442ee3ddbc9fec10f3ac259baaa5098f837d24b9bb19bbece9191ca6aa"} Nov 04 05:42:28.121497 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.121463 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj"] Nov 04 05:42:28.125489 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.125464 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" Nov 04 05:42:28.132679 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.132615 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj"] Nov 04 05:42:28.136509 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.136488 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-9d808e\"" Nov 04 05:42:28.224483 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.224445 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d9a7888-648f-441c-8f19-281ec23a306e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj\" (UID: \"0d9a7888-648f-441c-8f19-281ec23a306e\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" Nov 04 05:42:28.224650 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.224576 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr65x\" (UniqueName: \"kubernetes.io/projected/0d9a7888-648f-441c-8f19-281ec23a306e-kube-api-access-mr65x\") pod \"payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj\" (UID: \"0d9a7888-648f-441c-8f19-281ec23a306e\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" Nov 04 05:42:28.325135 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.325099 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d9a7888-648f-441c-8f19-281ec23a306e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj\" (UID: \"0d9a7888-648f-441c-8f19-281ec23a306e\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" Nov 04 05:42:28.325352 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.325177 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr65x\" (UniqueName: \"kubernetes.io/projected/0d9a7888-648f-441c-8f19-281ec23a306e-kube-api-access-mr65x\") pod \"payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj\" (UID: \"0d9a7888-648f-441c-8f19-281ec23a306e\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" Nov 04 05:42:28.325809 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.325787 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d9a7888-648f-441c-8f19-281ec23a306e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj\" (UID: \"0d9a7888-648f-441c-8f19-281ec23a306e\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" Nov 04 05:42:28.333654 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.333631 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr65x\" (UniqueName: \"kubernetes.io/projected/0d9a7888-648f-441c-8f19-281ec23a306e-kube-api-access-mr65x\") pod \"payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj\" (UID: \"0d9a7888-648f-441c-8f19-281ec23a306e\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" Nov 04 05:42:28.442306 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.442277 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" Nov 04 05:42:28.715275 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.715248 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj"] Nov 04 05:42:28.718451 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:42:28.718423 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d9a7888_648f_441c_8f19_281ec23a306e.slice/crio-ca3a8bc51c4566e6d9601ed3d5b161d646ee9912952b199e69bea4006b63719e WatchSource:0}: Error finding container ca3a8bc51c4566e6d9601ed3d5b161d646ee9912952b199e69bea4006b63719e: Status 404 returned error can't find the container with id ca3a8bc51c4566e6d9601ed3d5b161d646ee9912952b199e69bea4006b63719e Nov 04 05:42:28.728704 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.728670 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6"] Nov 04 05:42:28.732760 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.732738 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:42:28.735850 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.735828 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-2b14d0-serving-cert\"" Nov 04 05:42:28.742217 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.742169 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6"] Nov 04 05:42:28.769998 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.769965 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" event={"ID":"0d9a7888-648f-441c-8f19-281ec23a306e","Type":"ContainerStarted","Data":"ca3a8bc51c4566e6d9601ed3d5b161d646ee9912952b199e69bea4006b63719e"} Nov 04 05:42:28.771596 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.771571 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" event={"ID":"a9b1fde1-970f-4864-ac83-a2ce6110cbad","Type":"ContainerStarted","Data":"0db01921b4034377093fd224ad39a52a57d87c27543b6b46c8403ef78d76acc0"} Nov 04 05:42:28.771730 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.771690 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:42:28.794192 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.794138 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" podStartSLOduration=2.259850442 podStartE2EDuration="2.794124957s" totalImagesPullingTime="534.274515ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:42:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:42:28.792264429 +0000 UTC m=+795.749849379" watchObservedRunningTime="2026-11-04 05:42:28.794124957 +0000 UTC m=+795.751709925" Nov 04 05:42:28.829915 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.829878 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34aa7783-4df3-49a2-96f1-a0c801596082-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6\" (UID: \"34aa7783-4df3-49a2-96f1-a0c801596082\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:42:28.829915 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.829924 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34aa7783-4df3-49a2-96f1-a0c801596082-metrics-certs\") pod \"maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6\" (UID: \"34aa7783-4df3-49a2-96f1-a0c801596082\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:42:28.830213 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.829953 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34aa7783-4df3-49a2-96f1-a0c801596082-maas-api-tls\") pod \"maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6\" (UID: \"34aa7783-4df3-49a2-96f1-a0c801596082\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:42:28.830213 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.830160 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kq68\" (UniqueName: \"kubernetes.io/projected/34aa7783-4df3-49a2-96f1-a0c801596082-kube-api-access-8kq68\") pod \"maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6\" (UID: \"34aa7783-4df3-49a2-96f1-a0c801596082\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:42:28.931158 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.931123 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34aa7783-4df3-49a2-96f1-a0c801596082-metrics-certs\") pod \"maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6\" (UID: \"34aa7783-4df3-49a2-96f1-a0c801596082\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:42:28.931379 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.931166 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34aa7783-4df3-49a2-96f1-a0c801596082-maas-api-tls\") pod \"maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6\" (UID: \"34aa7783-4df3-49a2-96f1-a0c801596082\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:42:28.931379 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.931248 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kq68\" (UniqueName: \"kubernetes.io/projected/34aa7783-4df3-49a2-96f1-a0c801596082-kube-api-access-8kq68\") pod \"maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6\" (UID: \"34aa7783-4df3-49a2-96f1-a0c801596082\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:42:28.931379 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.931306 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34aa7783-4df3-49a2-96f1-a0c801596082-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6\" (UID: \"34aa7783-4df3-49a2-96f1-a0c801596082\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:42:28.931918 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.931898 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34aa7783-4df3-49a2-96f1-a0c801596082-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6\" (UID: \"34aa7783-4df3-49a2-96f1-a0c801596082\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:42:28.933671 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.933647 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34aa7783-4df3-49a2-96f1-a0c801596082-metrics-certs\") pod \"maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6\" (UID: \"34aa7783-4df3-49a2-96f1-a0c801596082\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:42:28.933759 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.933679 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34aa7783-4df3-49a2-96f1-a0c801596082-maas-api-tls\") pod \"maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6\" (UID: \"34aa7783-4df3-49a2-96f1-a0c801596082\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:42:28.938347 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:28.938329 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kq68\" (UniqueName: \"kubernetes.io/projected/34aa7783-4df3-49a2-96f1-a0c801596082-kube-api-access-8kq68\") pod \"maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6\" (UID: \"34aa7783-4df3-49a2-96f1-a0c801596082\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:42:29.054783 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.054696 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:42:29.338134 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.338103 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6"] Nov 04 05:42:29.603292 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.598587 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv"] Nov 04 05:42:29.605415 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.605390 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:42:29.611998 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.611966 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv"] Nov 04 05:42:29.612419 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.612269 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-9a0c8b-serving-cert\"" Nov 04 05:42:29.641123 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.641078 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e636dd2-3748-4814-a764-b14a51a0526a-maas-api-tls\") pod \"maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv\" (UID: \"3e636dd2-3748-4814-a764-b14a51a0526a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:42:29.641296 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.641142 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bllsw\" (UniqueName: \"kubernetes.io/projected/3e636dd2-3748-4814-a764-b14a51a0526a-kube-api-access-bllsw\") pod \"maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv\" (UID: \"3e636dd2-3748-4814-a764-b14a51a0526a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:42:29.641296 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.641278 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e636dd2-3748-4814-a764-b14a51a0526a-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv\" (UID: \"3e636dd2-3748-4814-a764-b14a51a0526a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:42:29.641418 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.641404 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e636dd2-3748-4814-a764-b14a51a0526a-metrics-certs\") pod \"maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv\" (UID: \"3e636dd2-3748-4814-a764-b14a51a0526a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:42:29.742310 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.742218 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e636dd2-3748-4814-a764-b14a51a0526a-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv\" (UID: \"3e636dd2-3748-4814-a764-b14a51a0526a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:42:29.742518 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.742381 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e636dd2-3748-4814-a764-b14a51a0526a-metrics-certs\") pod \"maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv\" (UID: \"3e636dd2-3748-4814-a764-b14a51a0526a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:42:29.742518 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.742470 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e636dd2-3748-4814-a764-b14a51a0526a-maas-api-tls\") pod \"maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv\" (UID: \"3e636dd2-3748-4814-a764-b14a51a0526a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:42:29.742740 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.742523 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bllsw\" (UniqueName: \"kubernetes.io/projected/3e636dd2-3748-4814-a764-b14a51a0526a-kube-api-access-bllsw\") pod \"maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv\" (UID: \"3e636dd2-3748-4814-a764-b14a51a0526a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:42:29.742991 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.742969 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e636dd2-3748-4814-a764-b14a51a0526a-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv\" (UID: \"3e636dd2-3748-4814-a764-b14a51a0526a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:42:29.745749 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.745725 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e636dd2-3748-4814-a764-b14a51a0526a-metrics-certs\") pod \"maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv\" (UID: \"3e636dd2-3748-4814-a764-b14a51a0526a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:42:29.746673 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.746646 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e636dd2-3748-4814-a764-b14a51a0526a-maas-api-tls\") pod \"maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv\" (UID: \"3e636dd2-3748-4814-a764-b14a51a0526a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:42:29.754111 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.754083 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bllsw\" (UniqueName: \"kubernetes.io/projected/3e636dd2-3748-4814-a764-b14a51a0526a-kube-api-access-bllsw\") pod \"maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv\" (UID: \"3e636dd2-3748-4814-a764-b14a51a0526a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:42:29.782131 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.782048 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" event={"ID":"34aa7783-4df3-49a2-96f1-a0c801596082","Type":"ContainerStarted","Data":"0333e84425eee58ea5b0bcd0097d152a2122bac3eeea8f13c963e1604ebdb8b4"} Nov 04 05:42:29.929964 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:29.929932 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:42:30.242595 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.242519 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv"] Nov 04 05:42:30.247748 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:42:30.247713 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e636dd2_3748_4814_a764_b14a51a0526a.slice/crio-2ba58f09667f6b5fd53e0ddccb2a96abb5b81aab7cef76dc42388edd34e1261d WatchSource:0}: Error finding container 2ba58f09667f6b5fd53e0ddccb2a96abb5b81aab7cef76dc42388edd34e1261d: Status 404 returned error can't find the container with id 2ba58f09667f6b5fd53e0ddccb2a96abb5b81aab7cef76dc42388edd34e1261d Nov 04 05:42:30.398039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.398002 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m"] Nov 04 05:42:30.418345 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.418313 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m"] Nov 04 05:42:30.418508 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.418430 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:30.433414 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.433368 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-27935d-serving-cert\"" Nov 04 05:42:30.450318 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.450277 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-metrics-certs\") pod \"maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:30.450850 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.450826 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzk2q\" (UniqueName: \"kubernetes.io/projected/16013ec4-91d6-4997-ae3f-7545cba44f5c-kube-api-access-vzk2q\") pod \"maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:30.450987 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.450868 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16013ec4-91d6-4997-ae3f-7545cba44f5c-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:30.450987 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.450917 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-maas-api-tls\") pod \"maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:30.552451 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.552362 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-metrics-certs\") pod \"maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:30.554069 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.553685 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzk2q\" (UniqueName: \"kubernetes.io/projected/16013ec4-91d6-4997-ae3f-7545cba44f5c-kube-api-access-vzk2q\") pod \"maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:30.554069 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.553728 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16013ec4-91d6-4997-ae3f-7545cba44f5c-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:30.554069 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.553781 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-maas-api-tls\") pod \"maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:30.554069 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:42:30.553985 5904 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-serving-cert: secret "maas-api-e2e-worker-w0-27935d-serving-cert" not found Nov 04 05:42:30.554069 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:42:30.554055 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-maas-api-tls podName:16013ec4-91d6-4997-ae3f-7545cba44f5c nodeName:}" failed. No retries permitted until 2026-11-04 05:42:31.054032817 +0000 UTC m=+798.011617760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-maas-api-tls") pod "maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" (UID: "16013ec4-91d6-4997-ae3f-7545cba44f5c") : secret "maas-api-e2e-worker-w0-27935d-serving-cert" not found Nov 04 05:42:30.555296 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.555271 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16013ec4-91d6-4997-ae3f-7545cba44f5c-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:30.556389 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.556366 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-metrics-certs\") pod \"maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:30.565021 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.564996 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzk2q\" (UniqueName: \"kubernetes.io/projected/16013ec4-91d6-4997-ae3f-7545cba44f5c-kube-api-access-vzk2q\") pod \"maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:30.791723 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.791676 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" event={"ID":"3e636dd2-3748-4814-a764-b14a51a0526a","Type":"ContainerStarted","Data":"fa5afcf7d82fdd43d194b2b1d9cd1f626f73b6c1b7f7f8d229e81afd27fc8b71"} Nov 04 05:42:30.791896 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.791757 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" event={"ID":"3e636dd2-3748-4814-a764-b14a51a0526a","Type":"ContainerStarted","Data":"2ba58f09667f6b5fd53e0ddccb2a96abb5b81aab7cef76dc42388edd34e1261d"} Nov 04 05:42:30.791896 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.791817 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:42:30.794292 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.794255 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" event={"ID":"34aa7783-4df3-49a2-96f1-a0c801596082","Type":"ContainerStarted","Data":"19f95d7d6d64a26549d0f1500b28a41f5c99b8fc9f2046a4cc0a6739cdbf9003"} Nov 04 05:42:30.794894 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.794869 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:42:30.809959 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:30.809919 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" podStartSLOduration=1.381263494 podStartE2EDuration="1.809905728s" totalImagesPullingTime="428.642234ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:42:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:42:30.807364761 +0000 UTC m=+797.764949711" watchObservedRunningTime="2026-11-04 05:42:30.809905728 +0000 UTC m=+797.767490676" Nov 04 05:42:31.060548 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:31.060474 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-maas-api-tls\") pod \"maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:31.078554 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:31.078521 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-maas-api-tls\") pod \"maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:31.343823 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:31.343726 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:31.643178 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:31.643134 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" podStartSLOduration=3.188912266 podStartE2EDuration="3.643120358s" totalImagesPullingTime="454.208092ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:42:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:42:30.824480539 +0000 UTC m=+797.782065490" watchObservedRunningTime="2026-11-04 05:42:31.643120358 +0000 UTC m=+798.600705308" Nov 04 05:42:31.643750 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:31.643732 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m"] Nov 04 05:42:31.645809 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:42:31.645780 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16013ec4_91d6_4997_ae3f_7545cba44f5c.slice/crio-3165e6b7b344c4802823e212485c0d508c8736c7e9482f11551ce2c5d7491c18 WatchSource:0}: Error finding container 3165e6b7b344c4802823e212485c0d508c8736c7e9482f11551ce2c5d7491c18: Status 404 returned error can't find the container with id 3165e6b7b344c4802823e212485c0d508c8736c7e9482f11551ce2c5d7491c18 Nov 04 05:42:31.800778 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:31.800743 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" event={"ID":"0d9a7888-648f-441c-8f19-281ec23a306e","Type":"ContainerStarted","Data":"9f1437507206c879a9137a008e6e8d8705cacdfbab3d6913fc3fe33e2c68eb67"} Nov 04 05:42:31.800973 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:31.800907 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" Nov 04 05:42:31.802012 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:31.801990 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" event={"ID":"16013ec4-91d6-4997-ae3f-7545cba44f5c","Type":"ContainerStarted","Data":"3165e6b7b344c4802823e212485c0d508c8736c7e9482f11551ce2c5d7491c18"} Nov 04 05:42:31.816330 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:31.816282 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" podStartSLOduration=1.16196758 podStartE2EDuration="3.816269595s" totalImagesPullingTime="2.654302015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:42:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:42:31.814562126 +0000 UTC m=+798.772147076" watchObservedRunningTime="2026-11-04 05:42:31.816269595 +0000 UTC m=+798.773854545" Nov 04 05:42:32.809312 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:32.809270 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" event={"ID":"16013ec4-91d6-4997-ae3f-7545cba44f5c","Type":"ContainerStarted","Data":"55ee4dfea074461831feebc85b648430c0d4ebfb75f3ed32483569238cb27e14"} Nov 04 05:42:32.809770 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:32.809521 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:32.829038 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:32.828983 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" podStartSLOduration=2.464620498 podStartE2EDuration="2.82896672s" totalImagesPullingTime="364.346222ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:42:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:42:32.827836712 +0000 UTC m=+799.785421661" watchObservedRunningTime="2026-11-04 05:42:32.82896672 +0000 UTC m=+799.786551670" Nov 04 05:42:33.415643 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:33.415610 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s"] Nov 04 05:42:33.420185 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:33.420161 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:33.434938 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:33.433973 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s"] Nov 04 05:42:33.434938 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:33.434104 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-ade2e8-serving-cert\"" Nov 04 05:42:33.484293 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:33.484255 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-metrics-certs\") pod \"maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:33.484470 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:33.484306 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:33.484470 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:33.484367 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:33.484470 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:33.484409 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8zpz\" (UniqueName: \"kubernetes.io/projected/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-kube-api-access-c8zpz\") pod \"maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:33.585634 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:33.585596 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:33.585793 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:33.585643 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:33.585793 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:33.585669 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8zpz\" (UniqueName: \"kubernetes.io/projected/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-kube-api-access-c8zpz\") pod \"maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:33.585954 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:42:33.585801 5904 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-serving-cert: secret "maas-api-e2e-worker-w2-ade2e8-serving-cert" not found Nov 04 05:42:33.585954 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:42:33.585876 5904 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-maas-api-tls podName:3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0 nodeName:}" failed. No retries permitted until 2026-11-04 05:42:34.085854229 +0000 UTC m=+801.043439174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-maas-api-tls") pod "maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" (UID: "3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0") : secret "maas-api-e2e-worker-w2-ade2e8-serving-cert" not found Nov 04 05:42:33.585954 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:33.585902 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-metrics-certs\") pod \"maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:33.586295 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:33.586276 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:33.588265 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:33.588249 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-metrics-certs\") pod \"maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:33.593877 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:33.593854 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8zpz\" (UniqueName: \"kubernetes.io/projected/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-kube-api-access-c8zpz\") pod \"maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:34.090499 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:34.090462 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:34.092867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:34.092840 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:34.338290 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:34.338258 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:34.623747 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:34.623717 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s"] Nov 04 05:42:34.626037 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:42:34.626012 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a3ea7f5_dfb9_4bdf_99a4_0e5ebe1ff7e0.slice/crio-3c15c8a8b955adabda0ce6c185324e6fa5218d896aa4e0d6f1a7c68ee2594136 WatchSource:0}: Error finding container 3c15c8a8b955adabda0ce6c185324e6fa5218d896aa4e0d6f1a7c68ee2594136: Status 404 returned error can't find the container with id 3c15c8a8b955adabda0ce6c185324e6fa5218d896aa4e0d6f1a7c68ee2594136 Nov 04 05:42:34.787998 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:34.787971 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:42:34.823837 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:34.823797 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" event={"ID":"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0","Type":"ContainerStarted","Data":"3c15c8a8b955adabda0ce6c185324e6fa5218d896aa4e0d6f1a7c68ee2594136"} Nov 04 05:42:35.831904 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:35.831866 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" event={"ID":"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0","Type":"ContainerStarted","Data":"b14a12ccb83ccfc2117c63927413d62b3a17e76e4785e301cdbdb1daf3596a44"} Nov 04 05:42:35.832414 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:35.832074 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:35.849435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:35.849380 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" podStartSLOduration=2.446067951 podStartE2EDuration="2.849363398s" totalImagesPullingTime="403.295447ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:42:35.847913936 +0000 UTC m=+802.805498888" watchObservedRunningTime="2026-11-04 05:42:35.849363398 +0000 UTC m=+802.806948350" Nov 04 05:42:36.808347 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:36.808315 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:42:37.814714 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:37.814685 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:42:38.820969 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:38.820940 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:42:41.843724 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:41.843695 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:42:42.813540 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:42:42.813513 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" Nov 04 05:42:57.134769 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:42:57.134691 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:43:03.214016 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.213975 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl"] Nov 04 05:43:03.220807 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.220780 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.227107 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.227082 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-ade2e8\"/\"default-dockercfg-q5cbg\"" Nov 04 05:43:03.228302 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.228244 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl"] Nov 04 05:43:03.230491 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.230467 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-ade2e8\"/\"kube-root-ca.crt\"" Nov 04 05:43:03.230731 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.230707 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-ade2e8\"/\"openshift-service-ca.crt\"" Nov 04 05:43:03.246853 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.246100 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-ade2e8\"/\"simulator-w2-ade2e8-kserve-self-signed-certs\"" Nov 04 05:43:03.401346 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.401297 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ed2d661-d028-42da-a363-3d5051726a75-tls-certs\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.401346 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.401348 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-dshm\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.401569 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.401519 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-model-cache\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.401610 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.401570 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-tmp-dir\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.401659 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.401628 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-home\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.401659 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.401646 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hgxr\" (UniqueName: \"kubernetes.io/projected/0ed2d661-d028-42da-a363-3d5051726a75-kube-api-access-8hgxr\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.502511 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.502432 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ed2d661-d028-42da-a363-3d5051726a75-tls-certs\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.502511 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.502472 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-dshm\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.502710 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.502517 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-model-cache\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.502710 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.502549 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-tmp-dir\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.502710 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.502571 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-home\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.502710 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.502590 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hgxr\" (UniqueName: \"kubernetes.io/projected/0ed2d661-d028-42da-a363-3d5051726a75-kube-api-access-8hgxr\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.503029 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.502999 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-tmp-dir\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.503029 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.503024 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-model-cache\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.503315 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.503067 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-home\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.505476 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.505451 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-dshm\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.505737 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.505719 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ed2d661-d028-42da-a363-3d5051726a75-tls-certs\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.510543 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.510519 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hgxr\" (UniqueName: \"kubernetes.io/projected/0ed2d661-d028-42da-a363-3d5051726a75-kube-api-access-8hgxr\") pod \"simulator-w2-ade2e8-kserve-8568777968-998hl\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.536732 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.536705 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:03.825328 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.825292 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl"] Nov 04 05:43:03.826744 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:43:03.826713 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ed2d661_d028_42da_a363_3d5051726a75.slice/crio-a276ceb3cd87f77148f510660f7e163bd1a022dd90067489a65e23c26d44db56 WatchSource:0}: Error finding container a276ceb3cd87f77148f510660f7e163bd1a022dd90067489a65e23c26d44db56: Status 404 returned error can't find the container with id a276ceb3cd87f77148f510660f7e163bd1a022dd90067489a65e23c26d44db56 Nov 04 05:43:03.992390 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:03.992350 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" event={"ID":"0ed2d661-d028-42da-a363-3d5051726a75","Type":"ContainerStarted","Data":"a276ceb3cd87f77148f510660f7e163bd1a022dd90067489a65e23c26d44db56"} Nov 04 05:43:06.006764 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:06.006724 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" event={"ID":"0ed2d661-d028-42da-a363-3d5051726a75","Type":"ContainerStarted","Data":"5b627a01fc5c3794df11b48adb2020cc6249867569483f11d9e5bd55fbdb4a93"} Nov 04 05:43:06.023632 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:06.023584 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" podStartSLOduration=1.791088132 podStartE2EDuration="3.02357174s" totalImagesPullingTime="1.232483608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:43:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:43:06.021437125 +0000 UTC m=+832.979022103" watchObservedRunningTime="2026-11-04 05:43:06.02357174 +0000 UTC m=+832.981156689" Nov 04 05:43:06.537730 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:06.537694 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:07.769649 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.769610 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns"] Nov 04 05:43:07.774666 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.774642 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.779952 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.779913 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-f910f5\"/\"openshift-service-ca.crt\"" Nov 04 05:43:07.781943 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.781916 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns"] Nov 04 05:43:07.782659 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.782636 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-f910f5\"/\"default-dockercfg-nm5gm\"" Nov 04 05:43:07.786597 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.786579 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-f910f5\"/\"premium-w3-f910f5-kserve-self-signed-certs\"" Nov 04 05:43:07.843186 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.843144 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-tls-certs\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.843378 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.843203 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpf7j\" (UniqueName: \"kubernetes.io/projected/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-kube-api-access-zpf7j\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.843378 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.843253 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-home\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.843378 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.843327 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-tmp-dir\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.843489 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.843426 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-dshm\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.843523 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.843490 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-model-cache\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.879835 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.879799 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-f910f5\"/\"kube-root-ca.crt\"" Nov 04 05:43:07.944456 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.944421 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-tls-certs\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.944655 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.944479 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpf7j\" (UniqueName: \"kubernetes.io/projected/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-kube-api-access-zpf7j\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.944655 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.944519 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-home\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.944655 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.944543 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-tmp-dir\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.944655 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.944586 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-dshm\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.944655 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.944639 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-model-cache\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.945345 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.945310 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-model-cache\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.945522 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.945353 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-tmp-dir\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.945522 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.945461 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-home\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.947177 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.947152 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-dshm\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.947759 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.947737 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-tls-certs\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:07.952671 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:07.952647 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpf7j\" (UniqueName: \"kubernetes.io/projected/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-kube-api-access-zpf7j\") pod \"premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:08.090330 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:08.090286 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:08.374989 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:08.374966 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns"] Nov 04 05:43:08.376955 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:43:08.376915 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec3a08cf_c3a9_4f90_b6f9_9ce1a8f0cccf.slice/crio-c6483f8c222f538c3181b060e41ee5c1930e6d48f11ac93a9f72923d1d172ecd WatchSource:0}: Error finding container c6483f8c222f538c3181b060e41ee5c1930e6d48f11ac93a9f72923d1d172ecd: Status 404 returned error can't find the container with id c6483f8c222f538c3181b060e41ee5c1930e6d48f11ac93a9f72923d1d172ecd Nov 04 05:43:09.024964 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:09.024926 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" event={"ID":"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf","Type":"ContainerStarted","Data":"2b394d317c2caf3274e1eb68628ca93ab3714d9f3a2b65c6657dfcfe1e193eed"} Nov 04 05:43:09.025578 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:09.024971 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" event={"ID":"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf","Type":"ContainerStarted","Data":"c6483f8c222f538c3181b060e41ee5c1930e6d48f11ac93a9f72923d1d172ecd"} Nov 04 05:43:09.043158 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:09.043101 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" podStartSLOduration=2.043087473 podStartE2EDuration="2.043087473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:43:09.041290793 +0000 UTC m=+835.998875744" watchObservedRunningTime="2026-11-04 05:43:09.043087473 +0000 UTC m=+836.000672423" Nov 04 05:43:09.090522 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:09.090484 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:13.536880 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:13.536837 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:13.553334 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:13.553299 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:14.062641 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:14.062613 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:43:18.091107 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:18.091077 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:18.105152 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:18.105122 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:19.089964 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:19.089936 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:43:27.134958 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:43:27.134837 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:43:30.324528 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.324082 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5"] Nov 04 05:43:30.328701 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.328675 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.341268 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.341190 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5"] Nov 04 05:43:30.341595 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.341572 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-f9b4a9\"/\"default-dockercfg-j7xb5\"" Nov 04 05:43:30.341709 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.341672 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-f9b4a9\"/\"premium-w1-f9b4a9-kserve-self-signed-certs\"" Nov 04 05:43:30.341777 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.341739 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-f9b4a9\"/\"kube-root-ca.crt\"" Nov 04 05:43:30.341996 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.341792 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-f9b4a9\"/\"openshift-service-ca.crt\"" Nov 04 05:43:30.483991 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.483948 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-dshm\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.484205 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.484003 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-model-cache\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.484205 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.484095 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-tmp-dir\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.484205 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.484130 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j85rz\" (UniqueName: \"kubernetes.io/projected/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-kube-api-access-j85rz\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.484205 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.484201 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-home\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.484496 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.484261 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-tls-certs\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.585406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.585321 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j85rz\" (UniqueName: \"kubernetes.io/projected/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-kube-api-access-j85rz\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.585406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.585396 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-home\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.585638 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.585423 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-tls-certs\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.585638 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.585498 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-dshm\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.585638 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.585522 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-model-cache\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.585638 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.585614 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-tmp-dir\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.585917 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.585888 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-home\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.585979 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.585942 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-tmp-dir\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.586038 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.585994 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-model-cache\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.587775 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.587749 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-dshm\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.588136 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.588120 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-tls-certs\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.592655 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.592629 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j85rz\" (UniqueName: \"kubernetes.io/projected/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-kube-api-access-j85rz\") pod \"premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.644147 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.644111 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:30.929387 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:30.929334 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5"] Nov 04 05:43:30.932365 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:43:30.932340 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ef7cb54_a171_4839_aa34_d78e0b7bbe4b.slice/crio-f5a5ef906b20d14d555f50ddb791e2c6dec62f6afaabd0e5ac888cc6c8aed735 WatchSource:0}: Error finding container f5a5ef906b20d14d555f50ddb791e2c6dec62f6afaabd0e5ac888cc6c8aed735: Status 404 returned error can't find the container with id f5a5ef906b20d14d555f50ddb791e2c6dec62f6afaabd0e5ac888cc6c8aed735 Nov 04 05:43:31.150371 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:31.150336 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" event={"ID":"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b","Type":"ContainerStarted","Data":"572b04953f1ed37b32dc9b6a0c1017d69ee55eb337224b5bb1313e03c653bd16"} Nov 04 05:43:31.150371 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:31.150377 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" event={"ID":"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b","Type":"ContainerStarted","Data":"f5a5ef906b20d14d555f50ddb791e2c6dec62f6afaabd0e5ac888cc6c8aed735"} Nov 04 05:43:31.166979 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:31.166923 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" podStartSLOduration=1.1669045200000001 podStartE2EDuration="1.16690452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:43:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:43:31.165097459 +0000 UTC m=+858.122682410" watchObservedRunningTime="2026-11-04 05:43:31.16690452 +0000 UTC m=+858.124489471" Nov 04 05:43:31.645267 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:31.645204 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:40.644532 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:40.644499 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:40.657462 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:40.657430 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:41.215906 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:41.215880 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:43:51.638123 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.638083 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz"] Nov 04 05:43:51.642170 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.642144 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.648981 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.648953 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-9a0c8b-openshift-default-dockercfg-zrhdn\"" Nov 04 05:43:51.656339 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.656313 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz"] Nov 04 05:43:51.714041 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.714011 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-workload-socket\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.714302 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.714052 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-token\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.714302 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.714095 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-envoy\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.714302 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.714194 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-credential-socket\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.714302 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.714275 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-workload-certs\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.714527 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.714405 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-podinfo\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.714527 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.714446 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-data\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.714527 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.714482 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e250af86-5b3a-4ef2-879c-022bb361c11e-istiod-ca-cert\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.714685 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.714652 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6dwj\" (UniqueName: \"kubernetes.io/projected/e250af86-5b3a-4ef2-879c-022bb361c11e-kube-api-access-j6dwj\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.816185 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.816149 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-workload-socket\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.816185 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.816193 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-token\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.816435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.816248 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-envoy\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.816435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.816266 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-credential-socket\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.816435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.816292 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-workload-certs\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.816435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.816325 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-podinfo\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.816435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.816359 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-data\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.816435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.816390 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e250af86-5b3a-4ef2-879c-022bb361c11e-istiod-ca-cert\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.816751 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.816475 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6dwj\" (UniqueName: \"kubernetes.io/projected/e250af86-5b3a-4ef2-879c-022bb361c11e-kube-api-access-j6dwj\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.816751 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.816695 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-credential-socket\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.816751 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.816695 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-workload-socket\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.816912 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.816820 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-workload-certs\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.816912 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.816885 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-data\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.817166 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.817149 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e250af86-5b3a-4ef2-879c-022bb361c11e-istiod-ca-cert\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.818623 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.818596 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-envoy\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.818902 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.818880 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-podinfo\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.824054 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.824032 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-token\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.830581 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.830562 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6dwj\" (UniqueName: \"kubernetes.io/projected/e250af86-5b3a-4ef2-879c-022bb361c11e-kube-api-access-j6dwj\") pod \"e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:51.968633 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:51.968542 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:52.245512 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:52.245484 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz"] Nov 04 05:43:52.246938 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:43:52.246912 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode250af86_5b3a_4ef2_879c_022bb361c11e.slice/crio-532ddac38c5430213257fbc0ca16da7c93b3f727e4d7f20cfef0d53b53f442d5 WatchSource:0}: Error finding container 532ddac38c5430213257fbc0ca16da7c93b3f727e4d7f20cfef0d53b53f442d5: Status 404 returned error can't find the container with id 532ddac38c5430213257fbc0ca16da7c93b3f727e4d7f20cfef0d53b53f442d5 Nov 04 05:43:52.249303 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:52.249274 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:43:52.249401 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:52.249335 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:43:52.249401 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:52.249364 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:43:52.263666 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:52.263637 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" event={"ID":"e250af86-5b3a-4ef2-879c-022bb361c11e","Type":"ContainerStarted","Data":"532ddac38c5430213257fbc0ca16da7c93b3f727e4d7f20cfef0d53b53f442d5"} Nov 04 05:43:53.278171 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:53.278131 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" event={"ID":"e250af86-5b3a-4ef2-879c-022bb361c11e","Type":"ContainerStarted","Data":"38d30a7ddce3b9e0b59a74c675ff714c13784fb71a96a68efe27ba808459f3b3"} Nov 04 05:43:53.323831 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:53.323772 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" podStartSLOduration=2.323757616 podStartE2EDuration="2.323757616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:43:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:43:53.321533681 +0000 UTC m=+880.279118633" watchObservedRunningTime="2026-11-04 05:43:53.323757616 +0000 UTC m=+880.281342566" Nov 04 05:43:53.969458 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:53.969416 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:53.971216 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:53.971188 5904 patch_prober.go:29] interesting pod/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Nov 04 05:43:53.971393 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:53.971280 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" podUID="e250af86-5b3a-4ef2-879c-022bb361c11e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Nov 04 05:43:54.969424 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:54.969382 5904 patch_prober.go:29] interesting pod/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Nov 04 05:43:54.969816 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:54.969451 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" podUID="e250af86-5b3a-4ef2-879c-022bb361c11e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Nov 04 05:43:55.969652 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:55.969605 5904 patch_prober.go:29] interesting pod/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Nov 04 05:43:55.970150 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:55.969673 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" podUID="e250af86-5b3a-4ef2-879c-022bb361c11e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Nov 04 05:43:56.970250 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:56.969612 5904 patch_prober.go:29] interesting pod/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Nov 04 05:43:56.970250 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:56.969673 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" podUID="e250af86-5b3a-4ef2-879c-022bb361c11e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Nov 04 05:43:57.136120 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:43:57.136071 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:43:57.974699 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:57.971597 5904 patch_prober.go:29] interesting pod/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Nov 04 05:43:57.974699 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:57.971666 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" podUID="e250af86-5b3a-4ef2-879c-022bb361c11e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Nov 04 05:43:58.973404 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:58.973375 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:58.973771 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:58.973752 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:43:58.974541 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:43:58.974520 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:44:13.293887 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:13.293858 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-slrx2_82660df8-f278-4d7c-b02a-149527e30d27/ovn-acl-logging/0.log" Nov 04 05:44:13.295346 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:13.295319 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-slrx2_82660df8-f278-4d7c-b02a-149527e30d27/ovn-acl-logging/0.log" Nov 04 05:44:16.623201 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.623164 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm"] Nov 04 05:44:16.626155 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.626137 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.634267 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.634044 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-27935d\"/\"kube-root-ca.crt\"" Nov 04 05:44:16.634267 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.634202 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-27935d\"/\"distinct-w0-27935d-kserve-self-signed-certs\"" Nov 04 05:44:16.634533 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.634518 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-27935d\"/\"default-dockercfg-r78l2\"" Nov 04 05:44:16.634709 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.634674 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-27935d\"/\"openshift-service-ca.crt\"" Nov 04 05:44:16.636701 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.636680 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm"] Nov 04 05:44:16.699117 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.699083 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhlz6\" (UniqueName: \"kubernetes.io/projected/7c590b80-71fd-4378-bf0d-b296ad006152-kube-api-access-hhlz6\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.699344 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.699171 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c590b80-71fd-4378-bf0d-b296ad006152-tls-certs\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.699344 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.699247 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-dshm\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.699344 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.699284 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-model-cache\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.699531 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.699466 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-tmp-dir\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.699588 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.699552 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-home\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.800365 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.800323 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-dshm\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.800561 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.800385 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-model-cache\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.800561 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.800464 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-tmp-dir\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.800561 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.800511 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-home\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.800734 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.800624 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhlz6\" (UniqueName: \"kubernetes.io/projected/7c590b80-71fd-4378-bf0d-b296ad006152-kube-api-access-hhlz6\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.800734 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.800663 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c590b80-71fd-4378-bf0d-b296ad006152-tls-certs\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.800913 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.800887 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-model-cache\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.800985 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.800908 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-tmp-dir\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.801252 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.801219 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-home\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.803020 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.802981 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-dshm\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.803400 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.803382 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c590b80-71fd-4378-bf0d-b296ad006152-tls-certs\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.808859 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.808708 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhlz6\" (UniqueName: \"kubernetes.io/projected/7c590b80-71fd-4378-bf0d-b296ad006152-kube-api-access-hhlz6\") pod \"distinct-w0-27935d-kserve-64b84bf975-kwqwm\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:16.941626 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:16.941513 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:17.222300 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:17.222272 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm"] Nov 04 05:44:17.224464 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:44:17.224438 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c590b80_71fd_4378_bf0d_b296ad006152.slice/crio-cf9c2a882758182c7f50410429a77ff5da4355bd53f50dbad24d08d57824c93e WatchSource:0}: Error finding container cf9c2a882758182c7f50410429a77ff5da4355bd53f50dbad24d08d57824c93e: Status 404 returned error can't find the container with id cf9c2a882758182c7f50410429a77ff5da4355bd53f50dbad24d08d57824c93e Nov 04 05:44:17.420422 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:17.420383 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" event={"ID":"7c590b80-71fd-4378-bf0d-b296ad006152","Type":"ContainerStarted","Data":"7797d2187640256c4e277a52a70edbf20ace2aa31ac5c14cc7a5f79ff0b4493d"} Nov 04 05:44:17.420616 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:17.420431 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" event={"ID":"7c590b80-71fd-4378-bf0d-b296ad006152","Type":"ContainerStarted","Data":"cf9c2a882758182c7f50410429a77ff5da4355bd53f50dbad24d08d57824c93e"} Nov 04 05:44:17.437920 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:17.437861 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" podStartSLOduration=1.437847134 podStartE2EDuration="1.437847134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:17.436074276 +0000 UTC m=+904.393659227" watchObservedRunningTime="2026-11-04 05:44:17.437847134 +0000 UTC m=+904.395432108" Nov 04 05:44:17.942529 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:17.942488 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:22.136491 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.136455 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6561-4b0c8e"] Nov 04 05:44:22.139423 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.139402 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6561-4b0c8e" Nov 04 05:44:22.144634 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.144605 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-g4dw7\"" Nov 04 05:44:22.146027 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.145999 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6561-4b0c8e"] Nov 04 05:44:22.155957 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.155933 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6567-842238"] Nov 04 05:44:22.158470 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.158448 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6567-842238" Nov 04 05:44:22.163099 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.163076 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6567-842238"] Nov 04 05:44:22.261116 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.261075 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgcvl\" (UniqueName: \"kubernetes.io/projected/9a78eae7-6c8d-45ac-8696-10eb524292fc-kube-api-access-rgcvl\") pod \"test-curl-6567-842238\" (UID: \"9a78eae7-6c8d-45ac-8696-10eb524292fc\") " pod="opendatahub/test-curl-6567-842238" Nov 04 05:44:22.261320 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.261221 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scl5t\" (UniqueName: \"kubernetes.io/projected/77007a45-da21-4f1b-a593-2540b5f09593-kube-api-access-scl5t\") pod \"test-curl-6561-4b0c8e\" (UID: \"77007a45-da21-4f1b-a593-2540b5f09593\") " pod="opendatahub/test-curl-6561-4b0c8e" Nov 04 05:44:22.362511 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.362474 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scl5t\" (UniqueName: \"kubernetes.io/projected/77007a45-da21-4f1b-a593-2540b5f09593-kube-api-access-scl5t\") pod \"test-curl-6561-4b0c8e\" (UID: \"77007a45-da21-4f1b-a593-2540b5f09593\") " pod="opendatahub/test-curl-6561-4b0c8e" Nov 04 05:44:22.362680 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.362639 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgcvl\" (UniqueName: \"kubernetes.io/projected/9a78eae7-6c8d-45ac-8696-10eb524292fc-kube-api-access-rgcvl\") pod \"test-curl-6567-842238\" (UID: \"9a78eae7-6c8d-45ac-8696-10eb524292fc\") " pod="opendatahub/test-curl-6567-842238" Nov 04 05:44:22.370419 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.370394 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgcvl\" (UniqueName: \"kubernetes.io/projected/9a78eae7-6c8d-45ac-8696-10eb524292fc-kube-api-access-rgcvl\") pod \"test-curl-6567-842238\" (UID: \"9a78eae7-6c8d-45ac-8696-10eb524292fc\") " pod="opendatahub/test-curl-6567-842238" Nov 04 05:44:22.370597 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.370430 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scl5t\" (UniqueName: \"kubernetes.io/projected/77007a45-da21-4f1b-a593-2540b5f09593-kube-api-access-scl5t\") pod \"test-curl-6561-4b0c8e\" (UID: \"77007a45-da21-4f1b-a593-2540b5f09593\") " pod="opendatahub/test-curl-6561-4b0c8e" Nov 04 05:44:22.456736 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.456653 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6561-4b0c8e" Nov 04 05:44:22.472255 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.472209 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6567-842238" Nov 04 05:44:22.737292 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.737267 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6561-4b0c8e"] Nov 04 05:44:22.739720 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:44:22.739689 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77007a45_da21_4f1b_a593_2540b5f09593.slice/crio-9728bb0a40d3b2fe9de8a2ae19e28f43a2f761857204d656f9e93d98c8c00b22 WatchSource:0}: Error finding container 9728bb0a40d3b2fe9de8a2ae19e28f43a2f761857204d656f9e93d98c8c00b22: Status 404 returned error can't find the container with id 9728bb0a40d3b2fe9de8a2ae19e28f43a2f761857204d656f9e93d98c8c00b22 Nov 04 05:44:22.784043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:22.784019 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6567-842238"] Nov 04 05:44:22.786264 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:44:22.786201 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a78eae7_6c8d_45ac_8696_10eb524292fc.slice/crio-48dc2a78068cc932915eb5740868dd0975db3e0fe5a63d5b35748a508165f66b WatchSource:0}: Error finding container 48dc2a78068cc932915eb5740868dd0975db3e0fe5a63d5b35748a508165f66b: Status 404 returned error can't find the container with id 48dc2a78068cc932915eb5740868dd0975db3e0fe5a63d5b35748a508165f66b Nov 04 05:44:23.457490 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:23.457450 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6561-4b0c8e" event={"ID":"77007a45-da21-4f1b-a593-2540b5f09593","Type":"ContainerStarted","Data":"9728bb0a40d3b2fe9de8a2ae19e28f43a2f761857204d656f9e93d98c8c00b22"} Nov 04 05:44:23.458730 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:23.458705 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6567-842238" event={"ID":"9a78eae7-6c8d-45ac-8696-10eb524292fc","Type":"ContainerStarted","Data":"48dc2a78068cc932915eb5740868dd0975db3e0fe5a63d5b35748a508165f66b"} Nov 04 05:44:25.473035 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:25.472944 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6561-4b0c8e" event={"ID":"77007a45-da21-4f1b-a593-2540b5f09593","Type":"ContainerStarted","Data":"19e8e23b3e41c3f2360a30a133ef8dc1924cef5583af707bb35255ebb4d426e8"} Nov 04 05:44:25.488475 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:25.488418 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6561-4b0c8e" podStartSLOduration=1.04587414 podStartE2EDuration="3.488400085s" totalImagesPullingTime="2.442525945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:25.485066612 +0000 UTC m=+912.442651594" watchObservedRunningTime="2026-11-04 05:44:25.488400085 +0000 UTC m=+912.445985036" Nov 04 05:44:26.137139 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.137104 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6561-4b0c8e"] Nov 04 05:44:26.140879 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.140842 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6561-4b0c8e"] Nov 04 05:44:26.323548 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.323503 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx"] Nov 04 05:44:26.324206 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.324182 5904 cpu_manager.go:414] "containerMap: removing container" podUID="77007a45-da21-4f1b-a593-2540b5f09593" containerName="test-curl-6561-4b0c8e" Nov 04 05:44:26.324206 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.324204 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="77007a45-da21-4f1b-a593-2540b5f09593" containerName="test-curl-6561-4b0c8e" Nov 04 05:44:26.324452 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.324343 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="77007a45-da21-4f1b-a593-2540b5f09593" containerName="test-curl-6561-4b0c8e" Nov 04 05:44:26.331859 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.331837 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.338240 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.338204 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx"] Nov 04 05:44:26.339629 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.339501 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-2b14d0\"/\"default-dockercfg-l9x7p\"" Nov 04 05:44:26.339629 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.339538 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-2b14d0\"/\"openshift-service-ca.crt\"" Nov 04 05:44:26.339629 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.339587 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-2b14d0\"/\"distinct-w6-2b14d0-kserve-self-signed-certs\"" Nov 04 05:44:26.339629 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.339620 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-2b14d0\"/\"kube-root-ca.crt\"" Nov 04 05:44:26.409741 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.409651 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-dshm\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.409741 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.409692 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/106302a4-351e-43c6-a5c5-a032a085ccc4-tls-certs\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.409972 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.409816 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-model-cache\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.409972 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.409835 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64sm9\" (UniqueName: \"kubernetes.io/projected/106302a4-351e-43c6-a5c5-a032a085ccc4-kube-api-access-64sm9\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.410083 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.410040 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-home\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.410133 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.410088 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-tmp-dir\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.511268 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.511214 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/106302a4-351e-43c6-a5c5-a032a085ccc4-tls-certs\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.511733 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.511321 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-model-cache\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.511733 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.511353 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64sm9\" (UniqueName: \"kubernetes.io/projected/106302a4-351e-43c6-a5c5-a032a085ccc4-kube-api-access-64sm9\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.511733 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.511520 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-home\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.511733 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.511562 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-tmp-dir\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.511733 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.511591 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-dshm\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.512338 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.512312 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-model-cache\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.512502 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.512482 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-home\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.512565 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.512549 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-tmp-dir\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.514153 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.514128 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/106302a4-351e-43c6-a5c5-a032a085ccc4-tls-certs\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.514308 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.514241 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-dshm\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.519193 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.519156 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64sm9\" (UniqueName: \"kubernetes.io/projected/106302a4-351e-43c6-a5c5-a032a085ccc4-kube-api-access-64sm9\") pod \"distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.647182 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.647139 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:26.935959 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.935927 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx"] Nov 04 05:44:26.937575 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:44:26.937546 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod106302a4_351e_43c6_a5c5_a032a085ccc4.slice/crio-0f3468856b69d2d32f1aaf8f7113103ac7d929a30ca12ed060ce1dbd5f067728 WatchSource:0}: Error finding container 0f3468856b69d2d32f1aaf8f7113103ac7d929a30ca12ed060ce1dbd5f067728: Status 404 returned error can't find the container with id 0f3468856b69d2d32f1aaf8f7113103ac7d929a30ca12ed060ce1dbd5f067728 Nov 04 05:44:26.942511 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.942487 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:26.960109 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:26.960084 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:27.141704 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:44:27.141668 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:44:27.488177 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:27.488137 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" event={"ID":"106302a4-351e-43c6-a5c5-a032a085ccc4","Type":"ContainerStarted","Data":"e16a465c1dd85685eaa70e0b8790345683fbb7e68d133fe04b3f6a0deb89b695"} Nov 04 05:44:27.488177 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:27.488182 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" event={"ID":"106302a4-351e-43c6-a5c5-a032a085ccc4","Type":"ContainerStarted","Data":"0f3468856b69d2d32f1aaf8f7113103ac7d929a30ca12ed060ce1dbd5f067728"} Nov 04 05:44:27.488455 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:27.488426 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6561-4b0c8e" podUID="77007a45-da21-4f1b-a593-2540b5f09593" containerName="test-curl-6561-4b0c8e" containerID="cri-o://19e8e23b3e41c3f2360a30a133ef8dc1924cef5583af707bb35255ebb4d426e8" gracePeriod=2 Nov 04 05:44:27.500283 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:27.500262 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:44:27.505102 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:27.505063 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" podStartSLOduration=1.505050765 podStartE2EDuration="1.505050765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:27.503425508 +0000 UTC m=+914.461010459" watchObservedRunningTime="2026-11-04 05:44:27.505050765 +0000 UTC m=+914.462635737" Nov 04 05:44:27.647775 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:27.647738 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:29.702925 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:29.702893 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6561-4b0c8e" Nov 04 05:44:29.849675 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:29.849648 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77007a45-da21-4f1b-a593-2540b5f09593-kube-api-access-scl5t\" (UniqueName: \"kubernetes.io/projected/77007a45-da21-4f1b-a593-2540b5f09593-kube-api-access-scl5t\") pod \"77007a45-da21-4f1b-a593-2540b5f09593\" (UID: \"77007a45-da21-4f1b-a593-2540b5f09593\") " Nov 04 05:44:29.851688 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:29.851657 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77007a45-da21-4f1b-a593-2540b5f09593-kube-api-access-scl5t" pod "77007a45-da21-4f1b-a593-2540b5f09593" (UID: "77007a45-da21-4f1b-a593-2540b5f09593"). InnerVolumeSpecName "kube-api-access-scl5t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:29.950602 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:29.950573 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scl5t\" (UniqueName: \"kubernetes.io/projected/77007a45-da21-4f1b-a593-2540b5f09593-kube-api-access-scl5t\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:30.509875 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:30.509846 5904 generic.go:356] "Generic (PLEG): container finished" podID="77007a45-da21-4f1b-a593-2540b5f09593" containerID="19e8e23b3e41c3f2360a30a133ef8dc1924cef5583af707bb35255ebb4d426e8" exitCode=137 Nov 04 05:44:30.510062 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:30.509887 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6561-4b0c8e" Nov 04 05:44:30.510062 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:30.509937 5904 scope.go:122] "RemoveContainer" containerID="19e8e23b3e41c3f2360a30a133ef8dc1924cef5583af707bb35255ebb4d426e8" Nov 04 05:44:30.520052 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:30.520011 5904 scope.go:122] "RemoveContainer" containerID="19e8e23b3e41c3f2360a30a133ef8dc1924cef5583af707bb35255ebb4d426e8" Nov 04 05:44:30.520344 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:44:30.520325 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19e8e23b3e41c3f2360a30a133ef8dc1924cef5583af707bb35255ebb4d426e8\": container with ID starting with 19e8e23b3e41c3f2360a30a133ef8dc1924cef5583af707bb35255ebb4d426e8 not found: ID does not exist" containerID="19e8e23b3e41c3f2360a30a133ef8dc1924cef5583af707bb35255ebb4d426e8" Nov 04 05:44:30.520411 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:30.520353 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19e8e23b3e41c3f2360a30a133ef8dc1924cef5583af707bb35255ebb4d426e8"} err="failed to get container status \"19e8e23b3e41c3f2360a30a133ef8dc1924cef5583af707bb35255ebb4d426e8\": rpc error: code = NotFound desc = could not find container \"19e8e23b3e41c3f2360a30a133ef8dc1924cef5583af707bb35255ebb4d426e8\": container with ID starting with 19e8e23b3e41c3f2360a30a133ef8dc1924cef5583af707bb35255ebb4d426e8 not found: ID does not exist" Nov 04 05:44:30.520981 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:44:30.520960 5904 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6561-4b0c8e\" is forbidden: User \"system:node:ip-10-0-128-126.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-126.us-east-2.compute.internal' and this object" podUID="77007a45-da21-4f1b-a593-2540b5f09593" pod="opendatahub/test-curl-6561-4b0c8e" Nov 04 05:44:31.178377 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:31.178340 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="77007a45-da21-4f1b-a593-2540b5f09593" path="/var/lib/kubelet/pods/77007a45-da21-4f1b-a593-2540b5f09593/volumes" Nov 04 05:44:31.315138 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:31.315101 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl"] Nov 04 05:44:34.539752 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:34.539716 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6567-842238" event={"ID":"9a78eae7-6c8d-45ac-8696-10eb524292fc","Type":"ContainerStarted","Data":"b66d0208b282e4d0087bafd0d8bea40792d813180df0ba10dedc2561a19943fd"} Nov 04 05:44:34.554786 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:34.554726 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6567-842238" podStartSLOduration=1.4600359090000001 podStartE2EDuration="12.554708618s" totalImagesPullingTime="11.094672709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:34.552099502 +0000 UTC m=+921.509684453" watchObservedRunningTime="2026-11-04 05:44:34.554708618 +0000 UTC m=+921.512293573" Nov 04 05:44:35.220326 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:35.220294 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6567-842238"] Nov 04 05:44:35.223904 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:35.223875 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6567-842238"] Nov 04 05:44:36.552246 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:36.552171 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6567-842238" podUID="9a78eae7-6c8d-45ac-8696-10eb524292fc" containerName="test-curl-6567-842238" containerID="cri-o://b66d0208b282e4d0087bafd0d8bea40792d813180df0ba10dedc2561a19943fd" gracePeriod=2 Nov 04 05:44:36.554634 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:44:36.554599 5904 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6567-842238\" is forbidden: User \"system:node:ip-10-0-128-126.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-126.us-east-2.compute.internal' and this object" podUID="9a78eae7-6c8d-45ac-8696-10eb524292fc" pod="opendatahub/test-curl-6567-842238" Nov 04 05:44:36.648160 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:36.648122 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:36.662538 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:36.662509 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:36.682700 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:44:36.682662 5904 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6567-842238\" is forbidden: User \"system:node:ip-10-0-128-126.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-126.us-east-2.compute.internal' and this object" podUID="9a78eae7-6c8d-45ac-8696-10eb524292fc" pod="opendatahub/test-curl-6567-842238" Nov 04 05:44:37.380558 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:37.380521 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5"] Nov 04 05:44:37.569408 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:37.569380 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:44:38.732418 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:38.732383 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6567-842238" Nov 04 05:44:38.848279 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:38.848239 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a78eae7-6c8d-45ac-8696-10eb524292fc-kube-api-access-rgcvl\" (UniqueName: \"kubernetes.io/projected/9a78eae7-6c8d-45ac-8696-10eb524292fc-kube-api-access-rgcvl\") pod \"9a78eae7-6c8d-45ac-8696-10eb524292fc\" (UID: \"9a78eae7-6c8d-45ac-8696-10eb524292fc\") " Nov 04 05:44:38.850329 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:38.850301 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a78eae7-6c8d-45ac-8696-10eb524292fc-kube-api-access-rgcvl" pod "9a78eae7-6c8d-45ac-8696-10eb524292fc" (UID: "9a78eae7-6c8d-45ac-8696-10eb524292fc"). InnerVolumeSpecName "kube-api-access-rgcvl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:38.949978 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:38.949946 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgcvl\" (UniqueName: \"kubernetes.io/projected/9a78eae7-6c8d-45ac-8696-10eb524292fc-kube-api-access-rgcvl\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:39.169387 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:39.169313 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a78eae7-6c8d-45ac-8696-10eb524292fc" path="/var/lib/kubelet/pods/9a78eae7-6c8d-45ac-8696-10eb524292fc/volumes" Nov 04 05:44:39.573076 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:39.573044 5904 generic.go:356] "Generic (PLEG): container finished" podID="9a78eae7-6c8d-45ac-8696-10eb524292fc" containerID="b66d0208b282e4d0087bafd0d8bea40792d813180df0ba10dedc2561a19943fd" exitCode=137 Nov 04 05:44:39.573301 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:39.573091 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6567-842238" Nov 04 05:44:39.573301 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:39.573146 5904 scope.go:122] "RemoveContainer" containerID="b66d0208b282e4d0087bafd0d8bea40792d813180df0ba10dedc2561a19943fd" Nov 04 05:44:39.582100 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:39.582081 5904 scope.go:122] "RemoveContainer" containerID="b66d0208b282e4d0087bafd0d8bea40792d813180df0ba10dedc2561a19943fd" Nov 04 05:44:39.582380 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:44:39.582358 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b66d0208b282e4d0087bafd0d8bea40792d813180df0ba10dedc2561a19943fd\": container with ID starting with b66d0208b282e4d0087bafd0d8bea40792d813180df0ba10dedc2561a19943fd not found: ID does not exist" containerID="b66d0208b282e4d0087bafd0d8bea40792d813180df0ba10dedc2561a19943fd" Nov 04 05:44:39.582487 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:39.582389 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b66d0208b282e4d0087bafd0d8bea40792d813180df0ba10dedc2561a19943fd"} err="failed to get container status \"b66d0208b282e4d0087bafd0d8bea40792d813180df0ba10dedc2561a19943fd\": rpc error: code = NotFound desc = could not find container \"b66d0208b282e4d0087bafd0d8bea40792d813180df0ba10dedc2561a19943fd\": container with ID starting with b66d0208b282e4d0087bafd0d8bea40792d813180df0ba10dedc2561a19943fd not found: ID does not exist" Nov 04 05:44:41.473646 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.473608 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm"] Nov 04 05:44:41.474425 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.474403 5904 cpu_manager.go:414] "containerMap: removing container" podUID="9a78eae7-6c8d-45ac-8696-10eb524292fc" containerName="test-curl-6567-842238" Nov 04 05:44:41.474495 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.474427 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a78eae7-6c8d-45ac-8696-10eb524292fc" containerName="test-curl-6567-842238" Nov 04 05:44:41.474607 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.474586 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a78eae7-6c8d-45ac-8696-10eb524292fc" containerName="test-curl-6567-842238" Nov 04 05:44:41.478051 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.478032 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.485612 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.485580 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-27935d\"/\"distinct-2-w0-27935d-kserve-self-signed-certs\"" Nov 04 05:44:41.487287 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.487215 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm"] Nov 04 05:44:41.581552 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.581519 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b56d92b-5e88-469d-85e0-c7193ac880d9-tls-certs\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.581736 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.581572 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-tmp-dir\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.581736 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.581651 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-dshm\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.581736 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.581696 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx4g4\" (UniqueName: \"kubernetes.io/projected/0b56d92b-5e88-469d-85e0-c7193ac880d9-kube-api-access-dx4g4\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.581900 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.581738 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-model-cache\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.581900 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.581865 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-home\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.682966 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.682928 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-home\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.683146 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.682992 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b56d92b-5e88-469d-85e0-c7193ac880d9-tls-certs\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.683146 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.683039 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-tmp-dir\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.683146 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.683072 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-dshm\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.683146 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.683136 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx4g4\" (UniqueName: \"kubernetes.io/projected/0b56d92b-5e88-469d-85e0-c7193ac880d9-kube-api-access-dx4g4\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.683370 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.683210 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-model-cache\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.683465 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.683438 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-home\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.683727 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.683703 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-model-cache\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.683817 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.683751 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-tmp-dir\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.685485 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.685457 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-dshm\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.685867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.685845 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b56d92b-5e88-469d-85e0-c7193ac880d9-tls-certs\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.691558 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.691535 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx4g4\" (UniqueName: \"kubernetes.io/projected/0b56d92b-5e88-469d-85e0-c7193ac880d9-kube-api-access-dx4g4\") pod \"distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:41.796613 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:41.796531 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:42.075531 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:42.075501 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm"] Nov 04 05:44:42.078898 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:44:42.078869 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b56d92b_5e88_469d_85e0_c7193ac880d9.slice/crio-59800962ace0728abd1d99518b43ff5699dd5d64cb6c448741edd7c933e48b1b WatchSource:0}: Error finding container 59800962ace0728abd1d99518b43ff5699dd5d64cb6c448741edd7c933e48b1b: Status 404 returned error can't find the container with id 59800962ace0728abd1d99518b43ff5699dd5d64cb6c448741edd7c933e48b1b Nov 04 05:44:42.594935 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:42.594892 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" event={"ID":"0b56d92b-5e88-469d-85e0-c7193ac880d9","Type":"ContainerStarted","Data":"1976aae86b272633df82be222e7fd881e00ad89576025eb9253a85e68668e9b4"} Nov 04 05:44:42.595357 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:42.594942 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" event={"ID":"0b56d92b-5e88-469d-85e0-c7193ac880d9","Type":"ContainerStarted","Data":"59800962ace0728abd1d99518b43ff5699dd5d64cb6c448741edd7c933e48b1b"} Nov 04 05:44:42.615489 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:42.615431 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" podStartSLOduration=1.6154158170000001 podStartE2EDuration="1.615415817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:42.612768058 +0000 UTC m=+929.570353009" watchObservedRunningTime="2026-11-04 05:44:42.615415817 +0000 UTC m=+929.573000793" Nov 04 05:44:42.797425 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:42.797387 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:46.537695 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.537651 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" podUID="0ed2d661-d028-42da-a363-3d5051726a75" containerName="main" containerID="cri-o://5b627a01fc5c3794df11b48adb2020cc6249867569483f11d9e5bd55fbdb4a93" gracePeriod=45 Nov 04 05:44:46.809063 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.809034 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:44:46.849364 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.849332 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-tmp-dir\") pod \"0ed2d661-d028-42da-a363-3d5051726a75\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " Nov 04 05:44:46.849538 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.849372 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-dshm\") pod \"0ed2d661-d028-42da-a363-3d5051726a75\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " Nov 04 05:44:46.849538 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.849420 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-home\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-home\") pod \"0ed2d661-d028-42da-a363-3d5051726a75\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " Nov 04 05:44:46.849538 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.849444 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0ed2d661-d028-42da-a363-3d5051726a75-tls-certs\" (UniqueName: \"kubernetes.io/secret/0ed2d661-d028-42da-a363-3d5051726a75-tls-certs\") pod \"0ed2d661-d028-42da-a363-3d5051726a75\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " Nov 04 05:44:46.849709 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.849563 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-model-cache\") pod \"0ed2d661-d028-42da-a363-3d5051726a75\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " Nov 04 05:44:46.849709 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.849632 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ed2d661-d028-42da-a363-3d5051726a75-kube-api-access-8hgxr\" (UniqueName: \"kubernetes.io/projected/0ed2d661-d028-42da-a363-3d5051726a75-kube-api-access-8hgxr\") pod \"0ed2d661-d028-42da-a363-3d5051726a75\" (UID: \"0ed2d661-d028-42da-a363-3d5051726a75\") " Nov 04 05:44:46.849709 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.849640 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-tmp-dir" pod "0ed2d661-d028-42da-a363-3d5051726a75" (UID: "0ed2d661-d028-42da-a363-3d5051726a75"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:46.849865 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.849828 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-model-cache" pod "0ed2d661-d028-42da-a363-3d5051726a75" (UID: "0ed2d661-d028-42da-a363-3d5051726a75"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:46.850022 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.849997 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-home" pod "0ed2d661-d028-42da-a363-3d5051726a75" (UID: "0ed2d661-d028-42da-a363-3d5051726a75"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:46.850174 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.850098 5904 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-model-cache\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:46.850174 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.850119 5904 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-tmp-dir\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:46.850174 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.850132 5904 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-home\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:46.852179 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.852094 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ed2d661-d028-42da-a363-3d5051726a75-kube-api-access-8hgxr" pod "0ed2d661-d028-42da-a363-3d5051726a75" (UID: "0ed2d661-d028-42da-a363-3d5051726a75"). InnerVolumeSpecName "kube-api-access-8hgxr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:46.852179 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.852128 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ed2d661-d028-42da-a363-3d5051726a75-tls-certs" pod "0ed2d661-d028-42da-a363-3d5051726a75" (UID: "0ed2d661-d028-42da-a363-3d5051726a75"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:46.852449 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.852327 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-dshm" pod "0ed2d661-d028-42da-a363-3d5051726a75" (UID: "0ed2d661-d028-42da-a363-3d5051726a75"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:46.951977 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.951930 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hgxr\" (UniqueName: \"kubernetes.io/projected/0ed2d661-d028-42da-a363-3d5051726a75-kube-api-access-8hgxr\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:46.951977 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.951966 5904 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ed2d661-d028-42da-a363-3d5051726a75-dshm\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:46.951977 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:46.951979 5904 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ed2d661-d028-42da-a363-3d5051726a75-tls-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:47.162457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.160486 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742"] Nov 04 05:44:47.162457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.161100 5904 cpu_manager.go:414] "containerMap: removing container" podUID="0ed2d661-d028-42da-a363-3d5051726a75" containerName="main" Nov 04 05:44:47.162457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.161115 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ed2d661-d028-42da-a363-3d5051726a75" containerName="main" Nov 04 05:44:47.162457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.161217 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ed2d661-d028-42da-a363-3d5051726a75" containerName="main" Nov 04 05:44:47.164651 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.163900 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.173384 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.173354 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-2b14d0\"/\"distinct-2-w6-2b14d0-kserve-self-signed-certs\"" Nov 04 05:44:47.181975 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.181946 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742"] Nov 04 05:44:47.256256 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.256205 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-tmp-dir\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.256256 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.256254 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-dshm\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.256526 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.256280 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-tls-certs\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.256526 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.256447 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52hs7\" (UniqueName: \"kubernetes.io/projected/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-kube-api-access-52hs7\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.256526 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.256482 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-home\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.256686 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.256549 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-model-cache\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.358064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.358016 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52hs7\" (UniqueName: \"kubernetes.io/projected/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-kube-api-access-52hs7\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.358297 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.358078 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-home\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.358297 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.358124 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-model-cache\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.358297 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.358264 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-tmp-dir\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.358297 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.358293 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-dshm\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.358524 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.358332 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-tls-certs\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.358580 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.358563 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-home\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.358714 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.358694 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-tmp-dir\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.358776 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.358734 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-model-cache\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.360758 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.360734 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-dshm\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.361544 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.361522 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-tls-certs\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.366309 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.366284 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52hs7\" (UniqueName: \"kubernetes.io/projected/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-kube-api-access-52hs7\") pod \"distinct-2-w6-2b14d0-kserve-5c44449889-kw742\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.489006 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.488911 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:47.631457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.631418 5904 generic.go:356] "Generic (PLEG): container finished" podID="0ed2d661-d028-42da-a363-3d5051726a75" containerID="5b627a01fc5c3794df11b48adb2020cc6249867569483f11d9e5bd55fbdb4a93" exitCode=0 Nov 04 05:44:47.631913 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.631505 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" event={"ID":"0ed2d661-d028-42da-a363-3d5051726a75","Type":"ContainerDied","Data":"5b627a01fc5c3794df11b48adb2020cc6249867569483f11d9e5bd55fbdb4a93"} Nov 04 05:44:47.631913 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.631565 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" event={"ID":"0ed2d661-d028-42da-a363-3d5051726a75","Type":"ContainerDied","Data":"a276ceb3cd87f77148f510660f7e163bd1a022dd90067489a65e23c26d44db56"} Nov 04 05:44:47.631913 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.631587 5904 scope.go:122] "RemoveContainer" containerID="5b627a01fc5c3794df11b48adb2020cc6249867569483f11d9e5bd55fbdb4a93" Nov 04 05:44:47.631913 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.631491 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl" Nov 04 05:44:47.642495 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.642474 5904 scope.go:122] "RemoveContainer" containerID="5b627a01fc5c3794df11b48adb2020cc6249867569483f11d9e5bd55fbdb4a93" Nov 04 05:44:47.642802 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:44:47.642781 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b627a01fc5c3794df11b48adb2020cc6249867569483f11d9e5bd55fbdb4a93\": container with ID starting with 5b627a01fc5c3794df11b48adb2020cc6249867569483f11d9e5bd55fbdb4a93 not found: ID does not exist" containerID="5b627a01fc5c3794df11b48adb2020cc6249867569483f11d9e5bd55fbdb4a93" Nov 04 05:44:47.642877 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.642810 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b627a01fc5c3794df11b48adb2020cc6249867569483f11d9e5bd55fbdb4a93"} err="failed to get container status \"5b627a01fc5c3794df11b48adb2020cc6249867569483f11d9e5bd55fbdb4a93\": rpc error: code = NotFound desc = could not find container \"5b627a01fc5c3794df11b48adb2020cc6249867569483f11d9e5bd55fbdb4a93\": container with ID starting with 5b627a01fc5c3794df11b48adb2020cc6249867569483f11d9e5bd55fbdb4a93 not found: ID does not exist" Nov 04 05:44:47.649040 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.649012 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl"] Nov 04 05:44:47.653971 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.653947 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-ade2e8/simulator-w2-ade2e8-kserve-8568777968-998hl"] Nov 04 05:44:47.773665 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:47.773638 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742"] Nov 04 05:44:47.775402 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:44:47.775372 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ebf1b07_2f78_41f9_9e9f_0b93e4c6e7ec.slice/crio-4544cb3539d14b10c54b805509294578b0c4116d61b514d5dc955f7c122f5838 WatchSource:0}: Error finding container 4544cb3539d14b10c54b805509294578b0c4116d61b514d5dc955f7c122f5838: Status 404 returned error can't find the container with id 4544cb3539d14b10c54b805509294578b0c4116d61b514d5dc955f7c122f5838 Nov 04 05:44:48.639658 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:48.639620 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" event={"ID":"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec","Type":"ContainerStarted","Data":"6913ee2f0139f5c999c3230fa1d8193a78abaf7dd5e1fc7eb665c1d672f51dd7"} Nov 04 05:44:48.639658 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:48.639663 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" event={"ID":"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec","Type":"ContainerStarted","Data":"4544cb3539d14b10c54b805509294578b0c4116d61b514d5dc955f7c122f5838"} Nov 04 05:44:48.682240 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:48.682156 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" podStartSLOduration=1.682136411 podStartE2EDuration="1.682136411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:48.67786377 +0000 UTC m=+935.635448721" watchObservedRunningTime="2026-11-04 05:44:48.682136411 +0000 UTC m=+935.639721362" Nov 04 05:44:49.169515 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:49.169480 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ed2d661-d028-42da-a363-3d5051726a75" path="/var/lib/kubelet/pods/0ed2d661-d028-42da-a363-3d5051726a75/volumes" Nov 04 05:44:49.489222 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:49.489149 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:51.354469 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.354431 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p"] Nov 04 05:44:51.358047 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.358016 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.371311 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.366852 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-f910f5-openshift-default-dockercfg-ttdqw\"" Nov 04 05:44:51.371311 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.366915 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p"] Nov 04 05:44:51.405760 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.405720 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b00d56d-9587-42b9-81a1-1d3cb9787842-istiod-ca-cert\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.405969 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.405806 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-credential-socket\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.405969 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.405839 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-data\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.405969 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.405891 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-podinfo\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.405969 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.405931 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-workload-socket\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.405969 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.405956 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-envoy\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.406316 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.406056 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-token\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.406316 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.406098 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-workload-certs\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.406316 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.406148 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzk62\" (UniqueName: \"kubernetes.io/projected/0b00d56d-9587-42b9-81a1-1d3cb9787842-kube-api-access-bzk62\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.507638 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.507594 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzk62\" (UniqueName: \"kubernetes.io/projected/0b00d56d-9587-42b9-81a1-1d3cb9787842-kube-api-access-bzk62\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.507829 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.507693 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b00d56d-9587-42b9-81a1-1d3cb9787842-istiod-ca-cert\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.507829 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.507752 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-credential-socket\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.507829 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.507777 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-data\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.507829 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.507820 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-podinfo\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.508073 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.507856 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-workload-socket\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.508073 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.507881 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-envoy\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.508073 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.507973 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-token\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.508073 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.508010 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-workload-certs\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.508547 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.508507 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-data\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.509357 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.509331 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b00d56d-9587-42b9-81a1-1d3cb9787842-istiod-ca-cert\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.509519 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.509393 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-workload-certs\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.509671 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.509507 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-credential-socket\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.509894 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.509866 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-workload-socket\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.512067 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.512041 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-envoy\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.512364 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.512347 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-podinfo\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.516624 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.516600 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzk62\" (UniqueName: \"kubernetes.io/projected/0b00d56d-9587-42b9-81a1-1d3cb9787842-kube-api-access-bzk62\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.516881 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.516863 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-token\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.687167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.687071 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:51.796983 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.796941 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:51.810592 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.810565 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:51.844950 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.844919 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw"] Nov 04 05:44:51.849057 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.849038 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:51.856957 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.856916 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-27935d-openshift-default-dockercfg-72clk\"" Nov 04 05:44:51.857863 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.857732 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw"] Nov 04 05:44:51.913042 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.913015 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-envoy\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:51.913201 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.913176 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqfvd\" (UniqueName: \"kubernetes.io/projected/353f0a70-20f0-4878-b7fa-20221f62ebc4-kube-api-access-dqfvd\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:51.913286 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.913208 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-token\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:51.913327 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.913315 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-credential-socket\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:51.913367 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.913338 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-workload-certs\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:51.913367 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.913355 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-workload-socket\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:51.913545 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.913513 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-podinfo\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:51.913632 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.913614 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-data\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:51.913696 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.913651 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/353f0a70-20f0-4878-b7fa-20221f62ebc4-istiod-ca-cert\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:51.964823 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.964749 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p"] Nov 04 05:44:51.966716 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:44:51.966685 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b00d56d_9587_42b9_81a1_1d3cb9787842.slice/crio-252ad03a61f4ed9f14818499b536e3ea645c3dfb6e990d9ed4ec59d1cebc1e7f WatchSource:0}: Error finding container 252ad03a61f4ed9f14818499b536e3ea645c3dfb6e990d9ed4ec59d1cebc1e7f: Status 404 returned error can't find the container with id 252ad03a61f4ed9f14818499b536e3ea645c3dfb6e990d9ed4ec59d1cebc1e7f Nov 04 05:44:51.969413 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.969373 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:44:51.969527 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.969452 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:44:51.969527 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:51.969492 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:44:52.014453 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.014416 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqfvd\" (UniqueName: \"kubernetes.io/projected/353f0a70-20f0-4878-b7fa-20221f62ebc4-kube-api-access-dqfvd\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.014598 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.014458 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-token\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.014598 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.014506 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-credential-socket\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.014598 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.014525 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-workload-certs\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.014598 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.014548 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-workload-socket\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.014825 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.014616 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-podinfo\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.014825 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.014666 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-data\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.014825 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.014692 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/353f0a70-20f0-4878-b7fa-20221f62ebc4-istiod-ca-cert\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.014825 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.014734 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-envoy\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.015027 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.014950 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-workload-certs\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.015086 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.015041 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-credential-socket\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.015140 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.015098 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-workload-socket\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.015140 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.015126 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-data\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.015684 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.015637 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/353f0a70-20f0-4878-b7fa-20221f62ebc4-istiod-ca-cert\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.016977 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.016954 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-envoy\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.018006 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.017985 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-podinfo\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.022640 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.022612 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-token\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.022741 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.022720 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqfvd\" (UniqueName: \"kubernetes.io/projected/353f0a70-20f0-4878-b7fa-20221f62ebc4-kube-api-access-dqfvd\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.165709 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.165671 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:52.401808 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.401771 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" podUID="4ef7cb54-a171-4839-aa34-d78e0b7bbe4b" containerName="main" containerID="cri-o://572b04953f1ed37b32dc9b6a0c1017d69ee55eb337224b5bb1313e03c653bd16" gracePeriod=45 Nov 04 05:44:52.449422 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.449390 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw"] Nov 04 05:44:52.459384 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:44:52.459349 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353f0a70_20f0_4878_b7fa_20221f62ebc4.slice/crio-85cc0fc908925fcf4c807e03f5742ba84f0e0ef6acd05dccac3e986bde5eb056 WatchSource:0}: Error finding container 85cc0fc908925fcf4c807e03f5742ba84f0e0ef6acd05dccac3e986bde5eb056: Status 404 returned error can't find the container with id 85cc0fc908925fcf4c807e03f5742ba84f0e0ef6acd05dccac3e986bde5eb056 Nov 04 05:44:52.508087 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.508054 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:44:52.508183 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.508125 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:44:52.508183 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.508153 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:44:52.676486 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.676342 5904 generic.go:356] "Generic (PLEG): container finished" podID="4ef7cb54-a171-4839-aa34-d78e0b7bbe4b" containerID="572b04953f1ed37b32dc9b6a0c1017d69ee55eb337224b5bb1313e03c653bd16" exitCode=0 Nov 04 05:44:52.676486 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.676435 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" event={"ID":"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b","Type":"ContainerDied","Data":"572b04953f1ed37b32dc9b6a0c1017d69ee55eb337224b5bb1313e03c653bd16"} Nov 04 05:44:52.680515 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.678902 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" event={"ID":"353f0a70-20f0-4878-b7fa-20221f62ebc4","Type":"ContainerStarted","Data":"f3a472abe60d6635427e9e78bce3afe26e220060e57ed1034439baeeb84b8406"} Nov 04 05:44:52.680515 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.678959 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" event={"ID":"353f0a70-20f0-4878-b7fa-20221f62ebc4","Type":"ContainerStarted","Data":"85cc0fc908925fcf4c807e03f5742ba84f0e0ef6acd05dccac3e986bde5eb056"} Nov 04 05:44:52.686769 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.686128 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" event={"ID":"0b00d56d-9587-42b9-81a1-1d3cb9787842","Type":"ContainerStarted","Data":"8fff9c9f288cbcbd0000c74ddd2bc4f8b9a1ca332d2390ecd4e97ecc60838467"} Nov 04 05:44:52.686769 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.686184 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" event={"ID":"0b00d56d-9587-42b9-81a1-1d3cb9787842","Type":"ContainerStarted","Data":"252ad03a61f4ed9f14818499b536e3ea645c3dfb6e990d9ed4ec59d1cebc1e7f"} Nov 04 05:44:52.687418 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.687106 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:52.730579 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.712736 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" podStartSLOduration=1.712714118 podStartE2EDuration="1.712714118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:52.69970521 +0000 UTC m=+939.657290161" watchObservedRunningTime="2026-11-04 05:44:52.712714118 +0000 UTC m=+939.670299068" Nov 04 05:44:52.734284 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.734012 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:44:52.737098 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.737040 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" podStartSLOduration=1.737024023 podStartE2EDuration="1.737024023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:52.731682779 +0000 UTC m=+939.689267730" watchObservedRunningTime="2026-11-04 05:44:52.737024023 +0000 UTC m=+939.694608973" Nov 04 05:44:52.740363 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.740330 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:44:52.835493 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.835459 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-model-cache\") pod \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " Nov 04 05:44:52.835690 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.835551 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-tmp-dir\") pod \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " Nov 04 05:44:52.835690 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.835656 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-dshm\") pod \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " Nov 04 05:44:52.835806 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.835702 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-home\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-home\") pod \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " Nov 04 05:44:52.835806 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.835781 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-tls-certs\" (UniqueName: \"kubernetes.io/secret/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-tls-certs\") pod \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " Nov 04 05:44:52.835902 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.835877 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-kube-api-access-j85rz\" (UniqueName: \"kubernetes.io/projected/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-kube-api-access-j85rz\") pod \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\" (UID: \"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b\") " Nov 04 05:44:52.835977 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.835950 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-tmp-dir" pod "4ef7cb54-a171-4839-aa34-d78e0b7bbe4b" (UID: "4ef7cb54-a171-4839-aa34-d78e0b7bbe4b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:52.836206 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.836180 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-home" pod "4ef7cb54-a171-4839-aa34-d78e0b7bbe4b" (UID: "4ef7cb54-a171-4839-aa34-d78e0b7bbe4b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:52.836461 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.836440 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-model-cache" pod "4ef7cb54-a171-4839-aa34-d78e0b7bbe4b" (UID: "4ef7cb54-a171-4839-aa34-d78e0b7bbe4b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:52.836578 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.836479 5904 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-home\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:52.836704 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.836690 5904 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-tmp-dir\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:52.839512 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.839491 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-dshm" pod "4ef7cb54-a171-4839-aa34-d78e0b7bbe4b" (UID: "4ef7cb54-a171-4839-aa34-d78e0b7bbe4b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:52.839898 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.839874 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-kube-api-access-j85rz" pod "4ef7cb54-a171-4839-aa34-d78e0b7bbe4b" (UID: "4ef7cb54-a171-4839-aa34-d78e0b7bbe4b"). InnerVolumeSpecName "kube-api-access-j85rz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:52.840323 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.840304 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-tls-certs" pod "4ef7cb54-a171-4839-aa34-d78e0b7bbe4b" (UID: "4ef7cb54-a171-4839-aa34-d78e0b7bbe4b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:52.937827 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.937793 5904 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-dshm\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:52.937827 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.937825 5904 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-tls-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:52.937827 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.937835 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j85rz\" (UniqueName: \"kubernetes.io/projected/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-kube-api-access-j85rz\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:52.938119 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:52.937844 5904 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b-model-cache\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:53.194744 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:53.194644 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:44:53.198167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:53.198123 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Nov 04 05:44:53.198469 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:53.198440 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" podUID="353f0a70-20f0-4878-b7fa-20221f62ebc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Nov 04 05:44:53.697322 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:53.692026 5904 patch_prober.go:29] interesting pod/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Nov 04 05:44:53.697322 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:53.692108 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" podUID="0b00d56d-9587-42b9-81a1-1d3cb9787842" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Nov 04 05:44:53.702290 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:53.702266 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" Nov 04 05:44:53.702545 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:53.702371 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5" event={"ID":"4ef7cb54-a171-4839-aa34-d78e0b7bbe4b","Type":"ContainerDied","Data":"f5a5ef906b20d14d555f50ddb791e2c6dec62f6afaabd0e5ac888cc6c8aed735"} Nov 04 05:44:53.702679 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:53.702668 5904 scope.go:122] "RemoveContainer" containerID="572b04953f1ed37b32dc9b6a0c1017d69ee55eb337224b5bb1313e03c653bd16" Nov 04 05:44:53.723021 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:53.722999 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5"] Nov 04 05:44:53.726239 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:53.726206 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-f9b4a9/premium-w1-f9b4a9-kserve-7bf58b4bdb-2l2z5"] Nov 04 05:44:54.184250 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:54.183503 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Nov 04 05:44:54.184250 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:54.183567 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" podUID="353f0a70-20f0-4878-b7fa-20221f62ebc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Nov 04 05:44:54.692248 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:54.688267 5904 patch_prober.go:29] interesting pod/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Nov 04 05:44:54.692248 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:54.688332 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" podUID="0b00d56d-9587-42b9-81a1-1d3cb9787842" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Nov 04 05:44:55.171249 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:55.170912 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Nov 04 05:44:55.171249 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:55.170972 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" podUID="353f0a70-20f0-4878-b7fa-20221f62ebc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Nov 04 05:44:55.195250 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:55.193922 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4ef7cb54-a171-4839-aa34-d78e0b7bbe4b" path="/var/lib/kubelet/pods/4ef7cb54-a171-4839-aa34-d78e0b7bbe4b/volumes" Nov 04 05:44:55.701271 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:55.700299 5904 patch_prober.go:29] interesting pod/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Nov 04 05:44:55.701271 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:55.700360 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" podUID="0b00d56d-9587-42b9-81a1-1d3cb9787842" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Nov 04 05:44:56.178247 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:56.166674 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Nov 04 05:44:56.178247 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:56.166754 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" podUID="353f0a70-20f0-4878-b7fa-20221f62ebc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Nov 04 05:44:56.688249 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:56.688037 5904 patch_prober.go:29] interesting pod/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Nov 04 05:44:56.688249 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:56.688100 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" podUID="0b00d56d-9587-42b9-81a1-1d3cb9787842" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Nov 04 05:44:57.138577 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:44:57.138515 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:44:57.178099 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:57.177674 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Nov 04 05:44:57.178099 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:57.177735 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" podUID="353f0a70-20f0-4878-b7fa-20221f62ebc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Nov 04 05:44:57.495299 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:57.494339 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:57.542673 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:57.542641 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:57.688689 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:57.688218 5904 patch_prober.go:29] interesting pod/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Nov 04 05:44:57.688689 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:57.688310 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" podUID="0b00d56d-9587-42b9-81a1-1d3cb9787842" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Nov 04 05:44:58.113689 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:58.113403 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:44:58.202123 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:58.200736 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Nov 04 05:44:58.202123 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:58.200797 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" podUID="353f0a70-20f0-4878-b7fa-20221f62ebc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Nov 04 05:44:58.689274 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:58.689218 5904 patch_prober.go:29] interesting pod/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Nov 04 05:44:58.689587 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:58.689539 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" podUID="0b00d56d-9587-42b9-81a1-1d3cb9787842" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Nov 04 05:44:59.166262 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:59.166184 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Nov 04 05:44:59.166461 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:59.166290 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" podUID="353f0a70-20f0-4878-b7fa-20221f62ebc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Nov 04 05:44:59.691748 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:59.691710 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:59.692260 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:59.692219 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:44:59.692872 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:44:59.692849 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:45:00.288906 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:00.288875 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:45:00.289297 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:00.289275 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:45:00.290037 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:00.290015 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:45:00.782013 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:00.781977 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s"] Nov 04 05:45:00.782823 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:00.782789 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" podUID="3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0" containerName="maas-api" containerID="cri-o://b14a12ccb83ccfc2117c63927413d62b3a17e76e4785e301cdbdb1daf3596a44" gracePeriod=30 Nov 04 05:45:01.178629 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.178174 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:45:01.258899 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.258860 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-maas-api-tls\") pod \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " Nov 04 05:45:01.259066 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.259023 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-kube-api-access-c8zpz\" (UniqueName: \"kubernetes.io/projected/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-kube-api-access-c8zpz\") pod \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " Nov 04 05:45:01.259152 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.259135 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-openshift-service-ca\") pod \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " Nov 04 05:45:01.259215 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.259177 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-metrics-certs\") pod \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\" (UID: \"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0\") " Nov 04 05:45:01.260553 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.260512 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-openshift-service-ca" pod "3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0" (UID: "3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:01.269527 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.267676 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-kube-api-access-c8zpz" pod "3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0" (UID: "3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0"). InnerVolumeSpecName "kube-api-access-c8zpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:01.269527 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.267780 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-metrics-certs" pod "3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0" (UID: "3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:01.269706 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.269614 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-maas-api-tls" pod "3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0" (UID: "3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:01.360919 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.360880 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8zpz\" (UniqueName: \"kubernetes.io/projected/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-kube-api-access-c8zpz\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:01.360919 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.360919 5904 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-openshift-service-ca\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:01.361184 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.360934 5904 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-metrics-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:01.361184 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.360947 5904 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0-maas-api-tls\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:01.931108 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.931074 5904 generic.go:356] "Generic (PLEG): container finished" podID="3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0" containerID="b14a12ccb83ccfc2117c63927413d62b3a17e76e4785e301cdbdb1daf3596a44" exitCode=0 Nov 04 05:45:01.931646 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.931129 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" Nov 04 05:45:01.931646 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.931260 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" event={"ID":"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0","Type":"ContainerDied","Data":"b14a12ccb83ccfc2117c63927413d62b3a17e76e4785e301cdbdb1daf3596a44"} Nov 04 05:45:01.931646 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.931312 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s" event={"ID":"3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0","Type":"ContainerDied","Data":"3c15c8a8b955adabda0ce6c185324e6fa5218d896aa4e0d6f1a7c68ee2594136"} Nov 04 05:45:01.931646 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.931345 5904 scope.go:122] "RemoveContainer" containerID="b14a12ccb83ccfc2117c63927413d62b3a17e76e4785e301cdbdb1daf3596a44" Nov 04 05:45:01.948464 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.948438 5904 scope.go:122] "RemoveContainer" containerID="b14a12ccb83ccfc2117c63927413d62b3a17e76e4785e301cdbdb1daf3596a44" Nov 04 05:45:01.948842 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:45:01.948817 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b14a12ccb83ccfc2117c63927413d62b3a17e76e4785e301cdbdb1daf3596a44\": container with ID starting with b14a12ccb83ccfc2117c63927413d62b3a17e76e4785e301cdbdb1daf3596a44 not found: ID does not exist" containerID="b14a12ccb83ccfc2117c63927413d62b3a17e76e4785e301cdbdb1daf3596a44" Nov 04 05:45:01.948961 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.948853 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b14a12ccb83ccfc2117c63927413d62b3a17e76e4785e301cdbdb1daf3596a44"} err="failed to get container status \"b14a12ccb83ccfc2117c63927413d62b3a17e76e4785e301cdbdb1daf3596a44\": rpc error: code = NotFound desc = could not find container \"b14a12ccb83ccfc2117c63927413d62b3a17e76e4785e301cdbdb1daf3596a44\": container with ID starting with b14a12ccb83ccfc2117c63927413d62b3a17e76e4785e301cdbdb1daf3596a44 not found: ID does not exist" Nov 04 05:45:01.968608 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.968548 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s"] Nov 04 05:45:01.970547 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:01.970519 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ade2e8-6b8b9798bb-zxl4s"] Nov 04 05:45:02.431258 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.431208 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv"] Nov 04 05:45:02.431488 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.431457 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" podUID="3e636dd2-3748-4814-a764-b14a51a0526a" containerName="maas-api" containerID="cri-o://fa5afcf7d82fdd43d194b2b1d9cd1f626f73b6c1b7f7f8d229e81afd27fc8b71" gracePeriod=30 Nov 04 05:45:02.760826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.760794 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:45:02.780816 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.780770 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e636dd2-3748-4814-a764-b14a51a0526a-kube-api-access-bllsw\" (UniqueName: \"kubernetes.io/projected/3e636dd2-3748-4814-a764-b14a51a0526a-kube-api-access-bllsw\") pod \"3e636dd2-3748-4814-a764-b14a51a0526a\" (UID: \"3e636dd2-3748-4814-a764-b14a51a0526a\") " Nov 04 05:45:02.781008 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.780903 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3e636dd2-3748-4814-a764-b14a51a0526a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e636dd2-3748-4814-a764-b14a51a0526a-metrics-certs\") pod \"3e636dd2-3748-4814-a764-b14a51a0526a\" (UID: \"3e636dd2-3748-4814-a764-b14a51a0526a\") " Nov 04 05:45:02.781008 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.780944 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3e636dd2-3748-4814-a764-b14a51a0526a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e636dd2-3748-4814-a764-b14a51a0526a-maas-api-tls\") pod \"3e636dd2-3748-4814-a764-b14a51a0526a\" (UID: \"3e636dd2-3748-4814-a764-b14a51a0526a\") " Nov 04 05:45:02.781524 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.781126 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3e636dd2-3748-4814-a764-b14a51a0526a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e636dd2-3748-4814-a764-b14a51a0526a-openshift-service-ca\") pod \"3e636dd2-3748-4814-a764-b14a51a0526a\" (UID: \"3e636dd2-3748-4814-a764-b14a51a0526a\") " Nov 04 05:45:02.782003 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.781969 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e636dd2-3748-4814-a764-b14a51a0526a-openshift-service-ca" pod "3e636dd2-3748-4814-a764-b14a51a0526a" (UID: "3e636dd2-3748-4814-a764-b14a51a0526a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:02.783901 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.783860 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e636dd2-3748-4814-a764-b14a51a0526a-metrics-certs" pod "3e636dd2-3748-4814-a764-b14a51a0526a" (UID: "3e636dd2-3748-4814-a764-b14a51a0526a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:02.784423 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.784402 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e636dd2-3748-4814-a764-b14a51a0526a-kube-api-access-bllsw" pod "3e636dd2-3748-4814-a764-b14a51a0526a" (UID: "3e636dd2-3748-4814-a764-b14a51a0526a"). InnerVolumeSpecName "kube-api-access-bllsw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:02.788834 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.788812 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e636dd2-3748-4814-a764-b14a51a0526a-maas-api-tls" pod "3e636dd2-3748-4814-a764-b14a51a0526a" (UID: "3e636dd2-3748-4814-a764-b14a51a0526a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:02.883240 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.883200 5904 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e636dd2-3748-4814-a764-b14a51a0526a-metrics-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:02.883387 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.883247 5904 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e636dd2-3748-4814-a764-b14a51a0526a-maas-api-tls\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:02.883387 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.883264 5904 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e636dd2-3748-4814-a764-b14a51a0526a-openshift-service-ca\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:02.883387 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.883278 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bllsw\" (UniqueName: \"kubernetes.io/projected/3e636dd2-3748-4814-a764-b14a51a0526a-kube-api-access-bllsw\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:02.939846 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.939809 5904 generic.go:356] "Generic (PLEG): container finished" podID="3e636dd2-3748-4814-a764-b14a51a0526a" containerID="fa5afcf7d82fdd43d194b2b1d9cd1f626f73b6c1b7f7f8d229e81afd27fc8b71" exitCode=0 Nov 04 05:45:02.940352 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.939862 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" Nov 04 05:45:02.940352 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.939889 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" event={"ID":"3e636dd2-3748-4814-a764-b14a51a0526a","Type":"ContainerDied","Data":"fa5afcf7d82fdd43d194b2b1d9cd1f626f73b6c1b7f7f8d229e81afd27fc8b71"} Nov 04 05:45:02.940352 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.939959 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv" event={"ID":"3e636dd2-3748-4814-a764-b14a51a0526a","Type":"ContainerDied","Data":"2ba58f09667f6b5fd53e0ddccb2a96abb5b81aab7cef76dc42388edd34e1261d"} Nov 04 05:45:02.940352 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.939991 5904 scope.go:122] "RemoveContainer" containerID="fa5afcf7d82fdd43d194b2b1d9cd1f626f73b6c1b7f7f8d229e81afd27fc8b71" Nov 04 05:45:02.950532 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.950490 5904 scope.go:122] "RemoveContainer" containerID="fa5afcf7d82fdd43d194b2b1d9cd1f626f73b6c1b7f7f8d229e81afd27fc8b71" Nov 04 05:45:02.950831 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:45:02.950803 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa5afcf7d82fdd43d194b2b1d9cd1f626f73b6c1b7f7f8d229e81afd27fc8b71\": container with ID starting with fa5afcf7d82fdd43d194b2b1d9cd1f626f73b6c1b7f7f8d229e81afd27fc8b71 not found: ID does not exist" containerID="fa5afcf7d82fdd43d194b2b1d9cd1f626f73b6c1b7f7f8d229e81afd27fc8b71" Nov 04 05:45:02.950932 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.950838 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa5afcf7d82fdd43d194b2b1d9cd1f626f73b6c1b7f7f8d229e81afd27fc8b71"} err="failed to get container status \"fa5afcf7d82fdd43d194b2b1d9cd1f626f73b6c1b7f7f8d229e81afd27fc8b71\": rpc error: code = NotFound desc = could not find container \"fa5afcf7d82fdd43d194b2b1d9cd1f626f73b6c1b7f7f8d229e81afd27fc8b71\": container with ID starting with fa5afcf7d82fdd43d194b2b1d9cd1f626f73b6c1b7f7f8d229e81afd27fc8b71 not found: ID does not exist" Nov 04 05:45:02.964982 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.964927 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv"] Nov 04 05:45:02.968540 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:02.968519 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-9a0c8b-6bd5b9798b-crjlv"] Nov 04 05:45:03.168826 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:03.168789 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0" path="/var/lib/kubelet/pods/3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0/volumes" Nov 04 05:45:03.169202 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:03.169189 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3e636dd2-3748-4814-a764-b14a51a0526a" path="/var/lib/kubelet/pods/3e636dd2-3748-4814-a764-b14a51a0526a/volumes" Nov 04 05:45:06.528200 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.528166 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g"] Nov 04 05:45:06.528910 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.528888 5904 cpu_manager.go:414] "containerMap: removing container" podUID="4ef7cb54-a171-4839-aa34-d78e0b7bbe4b" containerName="main" Nov 04 05:45:06.528996 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.528912 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ef7cb54-a171-4839-aa34-d78e0b7bbe4b" containerName="main" Nov 04 05:45:06.528996 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.528930 5904 cpu_manager.go:414] "containerMap: removing container" podUID="3e636dd2-3748-4814-a764-b14a51a0526a" containerName="maas-api" Nov 04 05:45:06.528996 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.528937 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e636dd2-3748-4814-a764-b14a51a0526a" containerName="maas-api" Nov 04 05:45:06.528996 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.528947 5904 cpu_manager.go:414] "containerMap: removing container" podUID="3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0" containerName="maas-api" Nov 04 05:45:06.528996 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.528956 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0" containerName="maas-api" Nov 04 05:45:06.529245 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.529084 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ef7cb54-a171-4839-aa34-d78e0b7bbe4b" containerName="main" Nov 04 05:45:06.529245 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.529098 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e636dd2-3748-4814-a764-b14a51a0526a" containerName="maas-api" Nov 04 05:45:06.529245 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.529107 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a3ea7f5-dfb9-4bdf-99a4-0e5ebe1ff7e0" containerName="maas-api" Nov 04 05:45:06.536448 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.536423 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.542534 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.542508 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-2b14d0-openshift-default-dockercfg-ccjw4\"" Nov 04 05:45:06.546152 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.546124 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g"] Nov 04 05:45:06.616406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.616367 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw"] Nov 04 05:45:06.621346 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.621325 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.625499 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.625470 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-podinfo\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.625615 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.625540 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-workload-socket\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.625615 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.625582 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-envoy\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.625733 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.625694 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt2lq\" (UniqueName: \"kubernetes.io/projected/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-kube-api-access-lt2lq\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.625733 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.625724 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-data\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.625852 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.625834 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-workload-certs\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.625907 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.625890 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-token\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.625958 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.625908 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istiod-ca-cert\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.626008 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.625989 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-credential-socket\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.632313 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.632290 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw"] Nov 04 05:45:06.642727 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.642690 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-9d808e-openshift-default-dockercfg-sqq5t\"" Nov 04 05:45:06.726917 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.726876 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-token\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.726917 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.726918 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istiod-ca-cert\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.727161 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.726945 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-credential-socket\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.727161 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.726980 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tlrn\" (UniqueName: \"kubernetes.io/projected/e2b7b8df-514a-43c7-b0a9-920332da4dc6-kube-api-access-9tlrn\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.727161 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.726984 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742"] Nov 04 05:45:06.727161 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.727003 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-workload-certs\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.727161 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.727050 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-podinfo\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.727484 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.727354 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-credential-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.727484 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.727402 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-workload-socket\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.727484 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.727431 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-podinfo\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.727484 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.727443 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-credential-socket\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.727484 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.727458 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-data\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.727764 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.727604 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-token\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.727764 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.727723 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-workload-socket\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.727764 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.727737 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-envoy\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.727764 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.727749 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istiod-ca-cert\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.727971 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.727788 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt2lq\" (UniqueName: \"kubernetes.io/projected/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-kube-api-access-lt2lq\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.727971 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.727824 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-data\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.727971 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.727881 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-workload-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.727971 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.727958 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-envoy\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.728381 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.727984 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istiod-ca-cert\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.728381 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.728057 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-workload-certs\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.728870 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.728843 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-data\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.729378 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.729339 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-workload-certs\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.730468 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.730440 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-podinfo\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.730665 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.730643 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-envoy\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.731621 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.731599 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx"] Nov 04 05:45:06.746688 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.746666 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-token\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.750261 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.750215 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt2lq\" (UniqueName: \"kubernetes.io/projected/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-kube-api-access-lt2lq\") pod \"e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.829817 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.829782 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-credential-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.829817 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.829821 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-podinfo\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.830010 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.829837 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-data\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.830051 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.830020 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-token\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.830124 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.830111 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-workload-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.830167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.830156 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-envoy\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.830220 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.830184 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istiod-ca-cert\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.830299 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.830256 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-data\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.830352 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.830305 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-credential-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.830407 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.830371 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tlrn\" (UniqueName: \"kubernetes.io/projected/e2b7b8df-514a-43c7-b0a9-920332da4dc6-kube-api-access-9tlrn\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.830463 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.830403 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-workload-certs\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.830518 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.830484 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-workload-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.830766 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.830736 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-workload-certs\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.830910 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.830801 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istiod-ca-cert\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.832456 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.832430 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-envoy\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.832565 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.832549 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-podinfo\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.837906 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.837886 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-token\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.838167 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.838149 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tlrn\" (UniqueName: \"kubernetes.io/projected/e2b7b8df-514a-43c7-b0a9-920332da4dc6-kube-api-access-9tlrn\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:06.887664 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.887632 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:06.935892 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:06.935860 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:07.172750 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:07.172727 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g"] Nov 04 05:45:07.174115 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:45:07.174087 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e0612d3_74c4_4d7c_9306_13ed78cf1b47.slice/crio-5f7975af176b7e7ae14204c95f274156380e02a00edb6fcfaae0aa1204ac6380 WatchSource:0}: Error finding container 5f7975af176b7e7ae14204c95f274156380e02a00edb6fcfaae0aa1204ac6380: Status 404 returned error can't find the container with id 5f7975af176b7e7ae14204c95f274156380e02a00edb6fcfaae0aa1204ac6380 Nov 04 05:45:07.176511 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:07.176482 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:07.176578 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:07.176546 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:07.176614 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:07.176576 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:07.249586 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:07.249551 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw"] Nov 04 05:45:07.254163 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:07.254128 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:07.254297 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:07.254220 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:07.254363 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:07.254304 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:08.012479 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:08.012444 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" event={"ID":"e2b7b8df-514a-43c7-b0a9-920332da4dc6","Type":"ContainerStarted","Data":"58c14dd957c5241fdd00d493cfcf99606fce04f839e6115972d633ab3f2dfd82"} Nov 04 05:45:08.012986 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:08.012523 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" event={"ID":"e2b7b8df-514a-43c7-b0a9-920332da4dc6","Type":"ContainerStarted","Data":"f8c4298971fa917a482b7889cc4c3686809c4e6cb96960656eaf09567ef99f2c"} Nov 04 05:45:08.014733 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:08.014698 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" event={"ID":"2e0612d3-74c4-4d7c-9306-13ed78cf1b47","Type":"ContainerStarted","Data":"8ed29c738a8b86394df6c2c72b2246af556e54f2394db250e24b367417602864"} Nov 04 05:45:08.014836 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:08.014754 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" event={"ID":"2e0612d3-74c4-4d7c-9306-13ed78cf1b47","Type":"ContainerStarted","Data":"5f7975af176b7e7ae14204c95f274156380e02a00edb6fcfaae0aa1204ac6380"} Nov 04 05:45:08.034867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:08.034821 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" podStartSLOduration=2.034805217 podStartE2EDuration="2.034805217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:08.032186806 +0000 UTC m=+954.989771758" watchObservedRunningTime="2026-11-04 05:45:08.034805217 +0000 UTC m=+954.992390167" Nov 04 05:45:08.891968 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:08.891938 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:08.892605 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:08.892311 5904 patch_prober.go:29] interesting pod/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Nov 04 05:45:08.892605 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:08.892360 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" podUID="2e0612d3-74c4-4d7c-9306-13ed78cf1b47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Nov 04 05:45:08.937357 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:08.937322 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:08.938835 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:08.938810 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Nov 04 05:45:08.938898 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:08.938860 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" podUID="e2b7b8df-514a-43c7-b0a9-920332da4dc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Nov 04 05:45:09.888733 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:09.888694 5904 patch_prober.go:29] interesting pod/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Nov 04 05:45:09.889411 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:09.889376 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" podUID="2e0612d3-74c4-4d7c-9306-13ed78cf1b47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Nov 04 05:45:09.937019 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:09.936984 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Nov 04 05:45:09.937195 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:09.937031 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" podUID="e2b7b8df-514a-43c7-b0a9-920332da4dc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Nov 04 05:45:10.926272 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:10.926239 5904 patch_prober.go:29] interesting pod/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Nov 04 05:45:10.926823 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:10.926796 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" podUID="2e0612d3-74c4-4d7c-9306-13ed78cf1b47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Nov 04 05:45:10.936533 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:10.936508 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Nov 04 05:45:10.936795 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:10.936768 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" podUID="e2b7b8df-514a-43c7-b0a9-920332da4dc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Nov 04 05:45:11.891672 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:11.891634 5904 patch_prober.go:29] interesting pod/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Nov 04 05:45:11.891962 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:11.891934 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" podUID="2e0612d3-74c4-4d7c-9306-13ed78cf1b47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Nov 04 05:45:11.936496 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:11.936465 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Nov 04 05:45:11.936990 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:11.936518 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" podUID="e2b7b8df-514a-43c7-b0a9-920332da4dc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Nov 04 05:45:12.898251 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:12.897681 5904 patch_prober.go:29] interesting pod/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Nov 04 05:45:12.898251 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:12.897748 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" podUID="2e0612d3-74c4-4d7c-9306-13ed78cf1b47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Nov 04 05:45:12.936762 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:12.936726 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Nov 04 05:45:12.937136 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:12.936778 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" podUID="e2b7b8df-514a-43c7-b0a9-920332da4dc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Nov 04 05:45:13.888041 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:13.888006 5904 patch_prober.go:29] interesting pod/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Nov 04 05:45:13.888195 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:13.888061 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" podUID="2e0612d3-74c4-4d7c-9306-13ed78cf1b47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Nov 04 05:45:13.936582 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:13.936544 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Nov 04 05:45:13.936783 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:13.936609 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" podUID="e2b7b8df-514a-43c7-b0a9-920332da4dc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Nov 04 05:45:14.940970 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:14.940938 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:14.941480 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:14.941277 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:14.942055 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:14.942037 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:45:14.964910 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:14.964831 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" podStartSLOduration=8.963273209 podStartE2EDuration="8.963273209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:08.073613476 +0000 UTC m=+955.031198426" watchObservedRunningTime="2026-11-04 05:45:14.963273209 +0000 UTC m=+961.920858159" Nov 04 05:45:14.978377 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:14.978344 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:14.979411 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:14.979378 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:14.980300 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:14.980279 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:15.218264 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:15.218130 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm"] Nov 04 05:45:15.221905 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:15.221870 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm"] Nov 04 05:45:15.435877 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:15.435837 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns"] Nov 04 05:45:20.746726 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:20.746637 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz"] Nov 04 05:45:20.747291 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:20.746849 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" podUID="e250af86-5b3a-4ef2-879c-022bb361c11e" containerName="istio-proxy" containerID="cri-o://38d30a7ddce3b9e0b59a74c675ff714c13784fb71a96a68efe27ba808459f3b3" gracePeriod=30 Nov 04 05:45:21.750710 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:21.750661 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" podUID="0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec" containerName="main" containerID="cri-o://6913ee2f0139f5c999c3230fa1d8193a78abaf7dd5e1fc7eb665c1d672f51dd7" gracePeriod=45 Nov 04 05:45:21.753681 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:21.753645 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" podUID="106302a4-351e-43c6-a5c5-a032a085ccc4" containerName="main" containerID="cri-o://e16a465c1dd85685eaa70e0b8790345683fbb7e68d133fe04b3f6a0deb89b695" gracePeriod=45 Nov 04 05:45:21.895387 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:21.895339 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4"] Nov 04 05:45:21.899963 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:21.899921 5904 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" podUID="0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": dial tcp 10.134.0.59:8000: connect: connection refused" Nov 04 05:45:21.925910 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:21.925879 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4"] Nov 04 05:45:21.926156 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:21.925975 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.028770 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.028684 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-token\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.028930 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.028775 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-data\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.028930 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.028808 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-workload-socket\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.028930 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.028876 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-envoy\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.028930 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.028907 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swfkg\" (UniqueName: \"kubernetes.io/projected/792505fe-6c68-4def-bd32-a41ca2a1fb34-kube-api-access-swfkg\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.029168 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.029037 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-credential-socket\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.029168 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.029077 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-podinfo\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.029168 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.029117 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-workload-certs\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.029168 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.029142 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/792505fe-6c68-4def-bd32-a41ca2a1fb34-istiod-ca-cert\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.075903 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.075877 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:45:22.078982 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.078947 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.130747 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-home\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-home\") pod \"106302a4-351e-43c6-a5c5-a032a085ccc4\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.130795 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-tls-certs\" (UniqueName: \"kubernetes.io/secret/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-tls-certs\") pod \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.130833 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/106302a4-351e-43c6-a5c5-a032a085ccc4-tls-certs\" (UniqueName: \"kubernetes.io/secret/106302a4-351e-43c6-a5c5-a032a085ccc4-tls-certs\") pod \"106302a4-351e-43c6-a5c5-a032a085ccc4\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.130887 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-dshm\") pod \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.130991 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-model-cache\") pod \"106302a4-351e-43c6-a5c5-a032a085ccc4\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.131031 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-model-cache\") pod \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.131097 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/106302a4-351e-43c6-a5c5-a032a085ccc4-kube-api-access-64sm9\" (UniqueName: \"kubernetes.io/projected/106302a4-351e-43c6-a5c5-a032a085ccc4-kube-api-access-64sm9\") pod \"106302a4-351e-43c6-a5c5-a032a085ccc4\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.131143 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-tmp-dir\") pod \"106302a4-351e-43c6-a5c5-a032a085ccc4\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.131171 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-tmp-dir\") pod \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.131205 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-kube-api-access-52hs7\" (UniqueName: \"kubernetes.io/projected/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-kube-api-access-52hs7\") pod \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.131267 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-home\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-home\") pod \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\" (UID: \"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec\") " Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.131320 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-dshm\") pod \"106302a4-351e-43c6-a5c5-a032a085ccc4\" (UID: \"106302a4-351e-43c6-a5c5-a032a085ccc4\") " Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.131580 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-token\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.131631 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-model-cache" pod "0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec" (UID: "0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.131638 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-data\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.131670 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-workload-socket\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.131746 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-envoy\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.131782 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swfkg\" (UniqueName: \"kubernetes.io/projected/792505fe-6c68-4def-bd32-a41ca2a1fb34-kube-api-access-swfkg\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.131826 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-home" pod "106302a4-351e-43c6-a5c5-a032a085ccc4" (UID: "106302a4-351e-43c6-a5c5-a032a085ccc4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.131939 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-credential-socket\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.131981 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-podinfo\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.132023 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-workload-certs\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.132046 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/792505fe-6c68-4def-bd32-a41ca2a1fb34-istiod-ca-cert\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.132117 5904 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-model-cache\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:22.132457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.132135 5904 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-home\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:22.136315 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.132834 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/792505fe-6c68-4def-bd32-a41ca2a1fb34-istiod-ca-cert\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.136315 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.134150 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-data\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.136315 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.134288 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-model-cache" pod "106302a4-351e-43c6-a5c5-a032a085ccc4" (UID: "106302a4-351e-43c6-a5c5-a032a085ccc4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:22.136315 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.134870 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-workload-socket\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.136315 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.134958 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-tmp-dir" pod "106302a4-351e-43c6-a5c5-a032a085ccc4" (UID: "106302a4-351e-43c6-a5c5-a032a085ccc4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:22.136315 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.135005 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-tmp-dir" pod "0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec" (UID: "0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:22.136315 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.135420 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-workload-certs\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.136315 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.135655 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-credential-socket\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.136315 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.135756 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/106302a4-351e-43c6-a5c5-a032a085ccc4-kube-api-access-64sm9" pod "106302a4-351e-43c6-a5c5-a032a085ccc4" (UID: "106302a4-351e-43c6-a5c5-a032a085ccc4"). InnerVolumeSpecName "kube-api-access-64sm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:22.136315 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.135775 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-home" pod "0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec" (UID: "0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:22.139086 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.139046 5904 generic.go:356] "Generic (PLEG): container finished" podID="106302a4-351e-43c6-a5c5-a032a085ccc4" containerID="e16a465c1dd85685eaa70e0b8790345683fbb7e68d133fe04b3f6a0deb89b695" exitCode=0 Nov 04 05:45:22.139220 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.139164 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" event={"ID":"106302a4-351e-43c6-a5c5-a032a085ccc4","Type":"ContainerDied","Data":"e16a465c1dd85685eaa70e0b8790345683fbb7e68d133fe04b3f6a0deb89b695"} Nov 04 05:45:22.139220 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.139199 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" event={"ID":"106302a4-351e-43c6-a5c5-a032a085ccc4","Type":"ContainerDied","Data":"0f3468856b69d2d32f1aaf8f7113103ac7d929a30ca12ed060ce1dbd5f067728"} Nov 04 05:45:22.139361 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.139220 5904 scope.go:122] "RemoveContainer" containerID="e16a465c1dd85685eaa70e0b8790345683fbb7e68d133fe04b3f6a0deb89b695" Nov 04 05:45:22.139424 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.139385 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx" Nov 04 05:45:22.140368 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.140323 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-tls-certs" pod "0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec" (UID: "0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:22.140768 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.140717 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-envoy\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.142336 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.142029 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-dshm" pod "0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec" (UID: "0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:22.142336 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.142070 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-podinfo\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.143070 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.142846 5904 generic.go:356] "Generic (PLEG): container finished" podID="0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec" containerID="6913ee2f0139f5c999c3230fa1d8193a78abaf7dd5e1fc7eb665c1d672f51dd7" exitCode=0 Nov 04 05:45:22.143070 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.142898 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-dshm" pod "106302a4-351e-43c6-a5c5-a032a085ccc4" (UID: "106302a4-351e-43c6-a5c5-a032a085ccc4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:22.143070 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.142971 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" Nov 04 05:45:22.143070 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.143001 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" event={"ID":"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec","Type":"ContainerDied","Data":"6913ee2f0139f5c999c3230fa1d8193a78abaf7dd5e1fc7eb665c1d672f51dd7"} Nov 04 05:45:22.143070 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.143028 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742" event={"ID":"0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec","Type":"ContainerDied","Data":"4544cb3539d14b10c54b805509294578b0c4116d61b514d5dc955f7c122f5838"} Nov 04 05:45:22.143810 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.143786 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swfkg\" (UniqueName: \"kubernetes.io/projected/792505fe-6c68-4def-bd32-a41ca2a1fb34-kube-api-access-swfkg\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.143890 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.142838 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/106302a4-351e-43c6-a5c5-a032a085ccc4-tls-certs" pod "106302a4-351e-43c6-a5c5-a032a085ccc4" (UID: "106302a4-351e-43c6-a5c5-a032a085ccc4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:22.145124 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.144689 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-kube-api-access-52hs7" pod "0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec" (UID: "0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec"). InnerVolumeSpecName "kube-api-access-52hs7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:22.145248 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.145130 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-token\") pod \"e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.163251 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.163212 5904 scope.go:122] "RemoveContainer" containerID="e16a465c1dd85685eaa70e0b8790345683fbb7e68d133fe04b3f6a0deb89b695" Nov 04 05:45:22.163616 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:45:22.163595 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e16a465c1dd85685eaa70e0b8790345683fbb7e68d133fe04b3f6a0deb89b695\": container with ID starting with e16a465c1dd85685eaa70e0b8790345683fbb7e68d133fe04b3f6a0deb89b695 not found: ID does not exist" containerID="e16a465c1dd85685eaa70e0b8790345683fbb7e68d133fe04b3f6a0deb89b695" Nov 04 05:45:22.163711 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.163628 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e16a465c1dd85685eaa70e0b8790345683fbb7e68d133fe04b3f6a0deb89b695"} err="failed to get container status \"e16a465c1dd85685eaa70e0b8790345683fbb7e68d133fe04b3f6a0deb89b695\": rpc error: code = NotFound desc = could not find container \"e16a465c1dd85685eaa70e0b8790345683fbb7e68d133fe04b3f6a0deb89b695\": container with ID starting with e16a465c1dd85685eaa70e0b8790345683fbb7e68d133fe04b3f6a0deb89b695 not found: ID does not exist" Nov 04 05:45:22.163711 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.163653 5904 scope.go:122] "RemoveContainer" containerID="6913ee2f0139f5c999c3230fa1d8193a78abaf7dd5e1fc7eb665c1d672f51dd7" Nov 04 05:45:22.175660 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.175640 5904 scope.go:122] "RemoveContainer" containerID="6913ee2f0139f5c999c3230fa1d8193a78abaf7dd5e1fc7eb665c1d672f51dd7" Nov 04 05:45:22.176035 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:45:22.176003 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6913ee2f0139f5c999c3230fa1d8193a78abaf7dd5e1fc7eb665c1d672f51dd7\": container with ID starting with 6913ee2f0139f5c999c3230fa1d8193a78abaf7dd5e1fc7eb665c1d672f51dd7 not found: ID does not exist" containerID="6913ee2f0139f5c999c3230fa1d8193a78abaf7dd5e1fc7eb665c1d672f51dd7" Nov 04 05:45:22.176118 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.176046 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6913ee2f0139f5c999c3230fa1d8193a78abaf7dd5e1fc7eb665c1d672f51dd7"} err="failed to get container status \"6913ee2f0139f5c999c3230fa1d8193a78abaf7dd5e1fc7eb665c1d672f51dd7\": rpc error: code = NotFound desc = could not find container \"6913ee2f0139f5c999c3230fa1d8193a78abaf7dd5e1fc7eb665c1d672f51dd7\": container with ID starting with 6913ee2f0139f5c999c3230fa1d8193a78abaf7dd5e1fc7eb665c1d672f51dd7 not found: ID does not exist" Nov 04 05:45:22.232834 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.232802 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52hs7\" (UniqueName: \"kubernetes.io/projected/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-kube-api-access-52hs7\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:22.232834 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.232829 5904 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-home\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:22.232834 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.232840 5904 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-dshm\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:22.233054 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.232849 5904 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-tls-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:22.233054 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.232858 5904 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/106302a4-351e-43c6-a5c5-a032a085ccc4-tls-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:22.233054 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.232866 5904 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-dshm\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:22.233054 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.232874 5904 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-model-cache\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:22.233054 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.232883 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64sm9\" (UniqueName: \"kubernetes.io/projected/106302a4-351e-43c6-a5c5-a032a085ccc4-kube-api-access-64sm9\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:22.233054 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.232893 5904 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/106302a4-351e-43c6-a5c5-a032a085ccc4-tmp-dir\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:22.233054 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.232901 5904 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec-tmp-dir\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:22.244251 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.244202 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:22.467976 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.467948 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx"] Nov 04 05:45:22.472467 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.472443 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-2b14d0/distinct-w6-2b14d0-kserve-79ccd77fbb-rdqgx"] Nov 04 05:45:22.482756 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.482724 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742"] Nov 04 05:45:22.484689 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.484668 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-2b14d0/distinct-2-w6-2b14d0-kserve-5c44449889-kw742"] Nov 04 05:45:22.533043 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.533021 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4"] Nov 04 05:45:22.534579 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:45:22.534553 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod792505fe_6c68_4def_bd32_a41ca2a1fb34.slice/crio-3108a41296314ad3bf491872153df5788dbfa4a06bfdd55236adf9f29af4a575 WatchSource:0}: Error finding container 3108a41296314ad3bf491872153df5788dbfa4a06bfdd55236adf9f29af4a575: Status 404 returned error can't find the container with id 3108a41296314ad3bf491872153df5788dbfa4a06bfdd55236adf9f29af4a575 Nov 04 05:45:22.536862 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.536831 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:22.536953 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.536892 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:22.536953 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:22.536922 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:23.155446 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:23.155409 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" event={"ID":"792505fe-6c68-4def-bd32-a41ca2a1fb34","Type":"ContainerStarted","Data":"a37b8db06f38ce129bd02e7746546b61a65a249afca3ea1ccd1eb9f1b62e293f"} Nov 04 05:45:23.156073 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:23.156028 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" event={"ID":"792505fe-6c68-4def-bd32-a41ca2a1fb34","Type":"ContainerStarted","Data":"3108a41296314ad3bf491872153df5788dbfa4a06bfdd55236adf9f29af4a575"} Nov 04 05:45:23.176462 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:23.176419 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec" path="/var/lib/kubelet/pods/0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec/volumes" Nov 04 05:45:23.176968 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:23.176945 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="106302a4-351e-43c6-a5c5-a032a085ccc4" path="/var/lib/kubelet/pods/106302a4-351e-43c6-a5c5-a032a085ccc4/volumes" Nov 04 05:45:23.178201 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:23.178154 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" podStartSLOduration=2.178142891 podStartE2EDuration="2.178142891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:23.17583547 +0000 UTC m=+970.133420423" watchObservedRunningTime="2026-11-04 05:45:23.178142891 +0000 UTC m=+970.135727842" Nov 04 05:45:23.244399 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:23.244372 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:23.247372 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:23.247342 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Nov 04 05:45:23.247590 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:23.247567 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" podUID="792505fe-6c68-4def-bd32-a41ca2a1fb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Nov 04 05:45:24.339042 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:24.338810 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Nov 04 05:45:24.339042 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:24.338869 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" podUID="792505fe-6c68-4def-bd32-a41ca2a1fb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Nov 04 05:45:25.275357 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:25.252975 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Nov 04 05:45:25.275357 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:25.253057 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" podUID="792505fe-6c68-4def-bd32-a41ca2a1fb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Nov 04 05:45:26.133909 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.133881 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:45:26.256258 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.248610 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Nov 04 05:45:26.256258 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.248669 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" podUID="792505fe-6c68-4def-bd32-a41ca2a1fb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Nov 04 05:45:26.296800 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.296763 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e250af86-5b3a-4ef2-879c-022bb361c11e-kube-api-access-j6dwj\" (UniqueName: \"kubernetes.io/projected/e250af86-5b3a-4ef2-879c-022bb361c11e-kube-api-access-j6dwj\") pod \"e250af86-5b3a-4ef2-879c-022bb361c11e\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " Nov 04 05:45:26.296981 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.296828 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-envoy\") pod \"e250af86-5b3a-4ef2-879c-022bb361c11e\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " Nov 04 05:45:26.296981 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.296873 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-credential-socket\") pod \"e250af86-5b3a-4ef2-879c-022bb361c11e\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " Nov 04 05:45:26.296981 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.296897 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-workload-certs\") pod \"e250af86-5b3a-4ef2-879c-022bb361c11e\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " Nov 04 05:45:26.296981 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.296926 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-podinfo\") pod \"e250af86-5b3a-4ef2-879c-022bb361c11e\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " Nov 04 05:45:26.297187 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.297104 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-workload-socket\") pod \"e250af86-5b3a-4ef2-879c-022bb361c11e\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " Nov 04 05:45:26.297187 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.297133 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-token\" (UniqueName: \"kubernetes.io/projected/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-token\") pod \"e250af86-5b3a-4ef2-879c-022bb361c11e\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " Nov 04 05:45:26.297187 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.297170 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e250af86-5b3a-4ef2-879c-022bb361c11e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e250af86-5b3a-4ef2-879c-022bb361c11e-istiod-ca-cert\") pod \"e250af86-5b3a-4ef2-879c-022bb361c11e\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " Nov 04 05:45:26.297382 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.297206 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-data\") pod \"e250af86-5b3a-4ef2-879c-022bb361c11e\" (UID: \"e250af86-5b3a-4ef2-879c-022bb361c11e\") " Nov 04 05:45:26.297382 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.297248 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-workload-certs" pod "e250af86-5b3a-4ef2-879c-022bb361c11e" (UID: "e250af86-5b3a-4ef2-879c-022bb361c11e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:26.297382 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.297270 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-credential-socket" pod "e250af86-5b3a-4ef2-879c-022bb361c11e" (UID: "e250af86-5b3a-4ef2-879c-022bb361c11e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:26.297539 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.297449 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-workload-socket" pod "e250af86-5b3a-4ef2-879c-022bb361c11e" (UID: "e250af86-5b3a-4ef2-879c-022bb361c11e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:26.297631 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.297613 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-data" pod "e250af86-5b3a-4ef2-879c-022bb361c11e" (UID: "e250af86-5b3a-4ef2-879c-022bb361c11e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:26.297766 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.297725 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e250af86-5b3a-4ef2-879c-022bb361c11e-istiod-ca-cert" pod "e250af86-5b3a-4ef2-879c-022bb361c11e" (UID: "e250af86-5b3a-4ef2-879c-022bb361c11e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:26.297847 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.297769 5904 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-workload-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:26.297847 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.297788 5904 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-data\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:26.297847 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.297804 5904 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-credential-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:26.297847 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.297817 5904 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-workload-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:26.299756 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.299711 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-envoy" pod "e250af86-5b3a-4ef2-879c-022bb361c11e" (UID: "e250af86-5b3a-4ef2-879c-022bb361c11e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:26.299756 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.299722 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-podinfo" pod "e250af86-5b3a-4ef2-879c-022bb361c11e" (UID: "e250af86-5b3a-4ef2-879c-022bb361c11e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:45:26.299908 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.299823 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e250af86-5b3a-4ef2-879c-022bb361c11e-kube-api-access-j6dwj" pod "e250af86-5b3a-4ef2-879c-022bb361c11e" (UID: "e250af86-5b3a-4ef2-879c-022bb361c11e"). InnerVolumeSpecName "kube-api-access-j6dwj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:26.299908 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.299845 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-token" pod "e250af86-5b3a-4ef2-879c-022bb361c11e" (UID: "e250af86-5b3a-4ef2-879c-022bb361c11e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:26.364443 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.361488 5904 generic.go:356] "Generic (PLEG): container finished" podID="e250af86-5b3a-4ef2-879c-022bb361c11e" containerID="38d30a7ddce3b9e0b59a74c675ff714c13784fb71a96a68efe27ba808459f3b3" exitCode=0 Nov 04 05:45:26.364443 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.361698 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" event={"ID":"e250af86-5b3a-4ef2-879c-022bb361c11e","Type":"ContainerDied","Data":"38d30a7ddce3b9e0b59a74c675ff714c13784fb71a96a68efe27ba808459f3b3"} Nov 04 05:45:26.364443 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.361754 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" event={"ID":"e250af86-5b3a-4ef2-879c-022bb361c11e","Type":"ContainerDied","Data":"532ddac38c5430213257fbc0ca16da7c93b3f727e4d7f20cfef0d53b53f442d5"} Nov 04 05:45:26.364443 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.361784 5904 scope.go:122] "RemoveContainer" containerID="38d30a7ddce3b9e0b59a74c675ff714c13784fb71a96a68efe27ba808459f3b3" Nov 04 05:45:26.364443 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.361936 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz" Nov 04 05:45:26.392781 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.392656 5904 scope.go:122] "RemoveContainer" containerID="38d30a7ddce3b9e0b59a74c675ff714c13784fb71a96a68efe27ba808459f3b3" Nov 04 05:45:26.393061 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:45:26.393029 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38d30a7ddce3b9e0b59a74c675ff714c13784fb71a96a68efe27ba808459f3b3\": container with ID starting with 38d30a7ddce3b9e0b59a74c675ff714c13784fb71a96a68efe27ba808459f3b3 not found: ID does not exist" containerID="38d30a7ddce3b9e0b59a74c675ff714c13784fb71a96a68efe27ba808459f3b3" Nov 04 05:45:26.393187 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.393073 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38d30a7ddce3b9e0b59a74c675ff714c13784fb71a96a68efe27ba808459f3b3"} err="failed to get container status \"38d30a7ddce3b9e0b59a74c675ff714c13784fb71a96a68efe27ba808459f3b3\": rpc error: code = NotFound desc = could not find container \"38d30a7ddce3b9e0b59a74c675ff714c13784fb71a96a68efe27ba808459f3b3\": container with ID starting with 38d30a7ddce3b9e0b59a74c675ff714c13784fb71a96a68efe27ba808459f3b3 not found: ID does not exist" Nov 04 05:45:26.398970 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.398944 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6dwj\" (UniqueName: \"kubernetes.io/projected/e250af86-5b3a-4ef2-879c-022bb361c11e-kube-api-access-j6dwj\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:26.399084 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.398974 5904 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-envoy\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:26.399084 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.398989 5904 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-podinfo\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:26.399084 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.399009 5904 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e250af86-5b3a-4ef2-879c-022bb361c11e-istio-token\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:26.399084 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.399021 5904 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e250af86-5b3a-4ef2-879c-022bb361c11e-istiod-ca-cert\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:26.414632 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.414600 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz"] Nov 04 05:45:26.421638 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:26.421611 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-9a0c8b-openshift-default-66f94db8c5-d2vkz"] Nov 04 05:45:27.141157 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:45:27.141126 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:45:27.180216 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:27.180186 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e250af86-5b3a-4ef2-879c-022bb361c11e" path="/var/lib/kubelet/pods/e250af86-5b3a-4ef2-879c-022bb361c11e/volumes" Nov 04 05:45:27.268252 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:27.261576 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Nov 04 05:45:27.268252 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:27.261637 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" podUID="792505fe-6c68-4def-bd32-a41ca2a1fb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Nov 04 05:45:28.251625 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:28.251589 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Nov 04 05:45:28.269373 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:28.269321 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" podUID="792505fe-6c68-4def-bd32-a41ca2a1fb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Nov 04 05:45:29.046506 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:29.046467 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj"] Nov 04 05:45:29.244750 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:29.244705 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Nov 04 05:45:29.244895 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:29.244778 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" podUID="792505fe-6c68-4def-bd32-a41ca2a1fb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Nov 04 05:45:29.385025 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:29.384990 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6"] Nov 04 05:45:29.385502 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:29.385260 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" podUID="34aa7783-4df3-49a2-96f1-a0c801596082" containerName="maas-api" containerID="cri-o://19f95d7d6d64a26549d0f1500b28a41f5c99b8fc9f2046a4cc0a6739cdbf9003" gracePeriod=30 Nov 04 05:45:29.799192 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:29.799162 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:45:29.948142 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:29.948112 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/34aa7783-4df3-49a2-96f1-a0c801596082-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34aa7783-4df3-49a2-96f1-a0c801596082-maas-api-tls\") pod \"34aa7783-4df3-49a2-96f1-a0c801596082\" (UID: \"34aa7783-4df3-49a2-96f1-a0c801596082\") " Nov 04 05:45:29.949775 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:29.948270 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/34aa7783-4df3-49a2-96f1-a0c801596082-metrics-certs\" (UniqueName: \"kubernetes.io/secret/34aa7783-4df3-49a2-96f1-a0c801596082-metrics-certs\") pod \"34aa7783-4df3-49a2-96f1-a0c801596082\" (UID: \"34aa7783-4df3-49a2-96f1-a0c801596082\") " Nov 04 05:45:29.949775 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:29.948585 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/34aa7783-4df3-49a2-96f1-a0c801596082-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34aa7783-4df3-49a2-96f1-a0c801596082-openshift-service-ca\") pod \"34aa7783-4df3-49a2-96f1-a0c801596082\" (UID: \"34aa7783-4df3-49a2-96f1-a0c801596082\") " Nov 04 05:45:29.949775 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:29.948689 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34aa7783-4df3-49a2-96f1-a0c801596082-kube-api-access-8kq68\" (UniqueName: \"kubernetes.io/projected/34aa7783-4df3-49a2-96f1-a0c801596082-kube-api-access-8kq68\") pod \"34aa7783-4df3-49a2-96f1-a0c801596082\" (UID: \"34aa7783-4df3-49a2-96f1-a0c801596082\") " Nov 04 05:45:29.950002 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:29.949816 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34aa7783-4df3-49a2-96f1-a0c801596082-openshift-service-ca" pod "34aa7783-4df3-49a2-96f1-a0c801596082" (UID: "34aa7783-4df3-49a2-96f1-a0c801596082"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:29.951258 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:29.951205 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34aa7783-4df3-49a2-96f1-a0c801596082-kube-api-access-8kq68" pod "34aa7783-4df3-49a2-96f1-a0c801596082" (UID: "34aa7783-4df3-49a2-96f1-a0c801596082"). InnerVolumeSpecName "kube-api-access-8kq68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:29.951964 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:29.951941 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34aa7783-4df3-49a2-96f1-a0c801596082-maas-api-tls" pod "34aa7783-4df3-49a2-96f1-a0c801596082" (UID: "34aa7783-4df3-49a2-96f1-a0c801596082"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:29.952051 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:29.951974 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34aa7783-4df3-49a2-96f1-a0c801596082-metrics-certs" pod "34aa7783-4df3-49a2-96f1-a0c801596082" (UID: "34aa7783-4df3-49a2-96f1-a0c801596082"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:30.050790 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.050697 5904 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34aa7783-4df3-49a2-96f1-a0c801596082-maas-api-tls\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.050790 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.050725 5904 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34aa7783-4df3-49a2-96f1-a0c801596082-metrics-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.050790 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.050736 5904 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34aa7783-4df3-49a2-96f1-a0c801596082-openshift-service-ca\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.050790 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.050745 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kq68\" (UniqueName: \"kubernetes.io/projected/34aa7783-4df3-49a2-96f1-a0c801596082-kube-api-access-8kq68\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.243515 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.243469 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" podUID="0b56d92b-5e88-469d-85e0-c7193ac880d9" containerName="main" containerID="cri-o://1976aae86b272633df82be222e7fd881e00ad89576025eb9253a85e68668e9b4" gracePeriod=45 Nov 04 05:45:30.247929 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.247888 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" podUID="7c590b80-71fd-4378-bf0d-b296ad006152" containerName="main" containerID="cri-o://7797d2187640256c4e277a52a70edbf20ace2aa31ac5c14cc7a5f79ff0b4493d" gracePeriod=45 Nov 04 05:45:30.450340 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.449655 5904 generic.go:356] "Generic (PLEG): container finished" podID="34aa7783-4df3-49a2-96f1-a0c801596082" containerID="19f95d7d6d64a26549d0f1500b28a41f5c99b8fc9f2046a4cc0a6739cdbf9003" exitCode=0 Nov 04 05:45:30.450340 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.449702 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" Nov 04 05:45:30.450340 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.449763 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" event={"ID":"34aa7783-4df3-49a2-96f1-a0c801596082","Type":"ContainerDied","Data":"19f95d7d6d64a26549d0f1500b28a41f5c99b8fc9f2046a4cc0a6739cdbf9003"} Nov 04 05:45:30.450340 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.449808 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6" event={"ID":"34aa7783-4df3-49a2-96f1-a0c801596082","Type":"ContainerDied","Data":"0333e84425eee58ea5b0bcd0097d152a2122bac3eeea8f13c963e1604ebdb8b4"} Nov 04 05:45:30.450340 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.449835 5904 scope.go:122] "RemoveContainer" containerID="19f95d7d6d64a26549d0f1500b28a41f5c99b8fc9f2046a4cc0a6739cdbf9003" Nov 04 05:45:30.458116 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.457924 5904 generic.go:356] "Generic (PLEG): container finished" podID="0b56d92b-5e88-469d-85e0-c7193ac880d9" containerID="1976aae86b272633df82be222e7fd881e00ad89576025eb9253a85e68668e9b4" exitCode=0 Nov 04 05:45:30.458116 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.458066 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" event={"ID":"0b56d92b-5e88-469d-85e0-c7193ac880d9","Type":"ContainerDied","Data":"1976aae86b272633df82be222e7fd881e00ad89576025eb9253a85e68668e9b4"} Nov 04 05:45:30.461005 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.460973 5904 generic.go:356] "Generic (PLEG): container finished" podID="7c590b80-71fd-4378-bf0d-b296ad006152" containerID="7797d2187640256c4e277a52a70edbf20ace2aa31ac5c14cc7a5f79ff0b4493d" exitCode=0 Nov 04 05:45:30.461159 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.461091 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" event={"ID":"7c590b80-71fd-4378-bf0d-b296ad006152","Type":"ContainerDied","Data":"7797d2187640256c4e277a52a70edbf20ace2aa31ac5c14cc7a5f79ff0b4493d"} Nov 04 05:45:30.470356 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.470310 5904 scope.go:122] "RemoveContainer" containerID="19f95d7d6d64a26549d0f1500b28a41f5c99b8fc9f2046a4cc0a6739cdbf9003" Nov 04 05:45:30.470755 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:45:30.470631 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19f95d7d6d64a26549d0f1500b28a41f5c99b8fc9f2046a4cc0a6739cdbf9003\": container with ID starting with 19f95d7d6d64a26549d0f1500b28a41f5c99b8fc9f2046a4cc0a6739cdbf9003 not found: ID does not exist" containerID="19f95d7d6d64a26549d0f1500b28a41f5c99b8fc9f2046a4cc0a6739cdbf9003" Nov 04 05:45:30.470755 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.470672 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19f95d7d6d64a26549d0f1500b28a41f5c99b8fc9f2046a4cc0a6739cdbf9003"} err="failed to get container status \"19f95d7d6d64a26549d0f1500b28a41f5c99b8fc9f2046a4cc0a6739cdbf9003\": rpc error: code = NotFound desc = could not find container \"19f95d7d6d64a26549d0f1500b28a41f5c99b8fc9f2046a4cc0a6739cdbf9003\": container with ID starting with 19f95d7d6d64a26549d0f1500b28a41f5c99b8fc9f2046a4cc0a6739cdbf9003 not found: ID does not exist" Nov 04 05:45:30.471572 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.471068 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" podUID="ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf" containerName="main" containerID="cri-o://2b394d317c2caf3274e1eb68628ca93ab3714d9f3a2b65c6657dfcfe1e193eed" gracePeriod=45 Nov 04 05:45:30.485310 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.485169 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6"] Nov 04 05:45:30.492199 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.491970 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-2b14d0-6749f7ff76-kzdf6"] Nov 04 05:45:30.598617 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.598590 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:45:30.603581 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.603540 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:45:30.656620 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.656582 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b56d92b-5e88-469d-85e0-c7193ac880d9-kube-api-access-dx4g4\" (UniqueName: \"kubernetes.io/projected/0b56d92b-5e88-469d-85e0-c7193ac880d9-kube-api-access-dx4g4\") pod \"0b56d92b-5e88-469d-85e0-c7193ac880d9\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " Nov 04 05:45:30.656873 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.656852 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0b56d92b-5e88-469d-85e0-c7193ac880d9-tls-certs\" (UniqueName: \"kubernetes.io/secret/0b56d92b-5e88-469d-85e0-c7193ac880d9-tls-certs\") pod \"0b56d92b-5e88-469d-85e0-c7193ac880d9\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " Nov 04 05:45:30.656978 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.656894 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-model-cache\") pod \"0b56d92b-5e88-469d-85e0-c7193ac880d9\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " Nov 04 05:45:30.656978 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.656952 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-home\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-home\") pod \"0b56d92b-5e88-469d-85e0-c7193ac880d9\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " Nov 04 05:45:30.657086 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.657030 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-dshm\") pod \"0b56d92b-5e88-469d-85e0-c7193ac880d9\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " Nov 04 05:45:30.657147 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.657125 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-tmp-dir\") pod \"0b56d92b-5e88-469d-85e0-c7193ac880d9\" (UID: \"0b56d92b-5e88-469d-85e0-c7193ac880d9\") " Nov 04 05:45:30.657699 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.657673 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-tmp-dir" pod "0b56d92b-5e88-469d-85e0-c7193ac880d9" (UID: "0b56d92b-5e88-469d-85e0-c7193ac880d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:30.658206 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.658176 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-model-cache" pod "0b56d92b-5e88-469d-85e0-c7193ac880d9" (UID: "0b56d92b-5e88-469d-85e0-c7193ac880d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:30.659147 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.659126 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-home" pod "0b56d92b-5e88-469d-85e0-c7193ac880d9" (UID: "0b56d92b-5e88-469d-85e0-c7193ac880d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:30.660806 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.660784 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b56d92b-5e88-469d-85e0-c7193ac880d9-tls-certs" pod "0b56d92b-5e88-469d-85e0-c7193ac880d9" (UID: "0b56d92b-5e88-469d-85e0-c7193ac880d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:30.661867 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.661831 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-dshm" pod "0b56d92b-5e88-469d-85e0-c7193ac880d9" (UID: "0b56d92b-5e88-469d-85e0-c7193ac880d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:30.662287 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.662263 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b56d92b-5e88-469d-85e0-c7193ac880d9-kube-api-access-dx4g4" pod "0b56d92b-5e88-469d-85e0-c7193ac880d9" (UID: "0b56d92b-5e88-469d-85e0-c7193ac880d9"). InnerVolumeSpecName "kube-api-access-dx4g4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:30.757783 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.757752 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-tmp-dir\") pod \"7c590b80-71fd-4378-bf0d-b296ad006152\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " Nov 04 05:45:30.757974 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.757809 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c590b80-71fd-4378-bf0d-b296ad006152-tls-certs\" (UniqueName: \"kubernetes.io/secret/7c590b80-71fd-4378-bf0d-b296ad006152-tls-certs\") pod \"7c590b80-71fd-4378-bf0d-b296ad006152\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " Nov 04 05:45:30.757974 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.757887 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-model-cache\") pod \"7c590b80-71fd-4378-bf0d-b296ad006152\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " Nov 04 05:45:30.757974 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.757916 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-home\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-home\") pod \"7c590b80-71fd-4378-bf0d-b296ad006152\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " Nov 04 05:45:30.758128 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.758010 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-dshm\") pod \"7c590b80-71fd-4378-bf0d-b296ad006152\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " Nov 04 05:45:30.758128 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.758054 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c590b80-71fd-4378-bf0d-b296ad006152-kube-api-access-hhlz6\" (UniqueName: \"kubernetes.io/projected/7c590b80-71fd-4378-bf0d-b296ad006152-kube-api-access-hhlz6\") pod \"7c590b80-71fd-4378-bf0d-b296ad006152\" (UID: \"7c590b80-71fd-4378-bf0d-b296ad006152\") " Nov 04 05:45:30.758128 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.758065 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-tmp-dir" pod "7c590b80-71fd-4378-bf0d-b296ad006152" (UID: "7c590b80-71fd-4378-bf0d-b296ad006152"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:30.758305 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.758144 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-model-cache" pod "7c590b80-71fd-4378-bf0d-b296ad006152" (UID: "7c590b80-71fd-4378-bf0d-b296ad006152"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:30.758371 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.758310 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-home" pod "7c590b80-71fd-4378-bf0d-b296ad006152" (UID: "7c590b80-71fd-4378-bf0d-b296ad006152"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:30.758543 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.758520 5904 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-tmp-dir\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.758543 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.758543 5904 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-dshm\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.758715 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.758556 5904 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-tmp-dir\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.758715 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.758568 5904 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-model-cache\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.758715 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.758581 5904 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-home\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.758715 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.758594 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx4g4\" (UniqueName: \"kubernetes.io/projected/0b56d92b-5e88-469d-85e0-c7193ac880d9-kube-api-access-dx4g4\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.758715 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.758605 5904 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b56d92b-5e88-469d-85e0-c7193ac880d9-tls-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.758715 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.758618 5904 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-model-cache\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.758715 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.758631 5904 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b56d92b-5e88-469d-85e0-c7193ac880d9-home\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.760314 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.760291 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c590b80-71fd-4378-bf0d-b296ad006152-tls-certs" pod "7c590b80-71fd-4378-bf0d-b296ad006152" (UID: "7c590b80-71fd-4378-bf0d-b296ad006152"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:30.760485 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.760451 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-dshm" pod "7c590b80-71fd-4378-bf0d-b296ad006152" (UID: "7c590b80-71fd-4378-bf0d-b296ad006152"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:30.760737 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.760719 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c590b80-71fd-4378-bf0d-b296ad006152-kube-api-access-hhlz6" pod "7c590b80-71fd-4378-bf0d-b296ad006152" (UID: "7c590b80-71fd-4378-bf0d-b296ad006152"). InnerVolumeSpecName "kube-api-access-hhlz6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:30.770922 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.770892 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:45:30.859330 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.859220 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-dshm\") pod \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " Nov 04 05:45:30.859620 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.859597 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-model-cache\") pod \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " Nov 04 05:45:30.859887 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.859863 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-tmp-dir\") pod \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " Nov 04 05:45:30.860004 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.859899 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-model-cache" pod "ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf" (UID: "ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:30.860004 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.859910 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-kube-api-access-zpf7j\" (UniqueName: \"kubernetes.io/projected/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-kube-api-access-zpf7j\") pod \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " Nov 04 05:45:30.860004 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.859985 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-tls-certs\" (UniqueName: \"kubernetes.io/secret/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-tls-certs\") pod \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " Nov 04 05:45:30.860169 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.860052 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-home\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-home\") pod \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\" (UID: \"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf\") " Nov 04 05:45:30.860261 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.860206 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-tmp-dir" pod "ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf" (UID: "ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:30.860923 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.860527 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-home" pod "ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf" (UID: "ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:30.860923 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.860815 5904 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-model-cache\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.860923 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.860836 5904 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c590b80-71fd-4378-bf0d-b296ad006152-dshm\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.860923 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.860854 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhlz6\" (UniqueName: \"kubernetes.io/projected/7c590b80-71fd-4378-bf0d-b296ad006152-kube-api-access-hhlz6\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.860923 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.860869 5904 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-tmp-dir\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.860923 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.860883 5904 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c590b80-71fd-4378-bf0d-b296ad006152-tls-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.860923 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.860897 5904 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-home\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.863069 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.863039 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-tls-certs" pod "ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf" (UID: "ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:30.863754 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.863723 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-kube-api-access-zpf7j" pod "ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf" (UID: "ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf"). InnerVolumeSpecName "kube-api-access-zpf7j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:30.864012 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.863982 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-dshm" pod "ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf" (UID: "ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:30.962209 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.962163 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpf7j\" (UniqueName: \"kubernetes.io/projected/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-kube-api-access-zpf7j\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.962209 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.962204 5904 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-tls-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.962494 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:30.962219 5904 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf-dshm\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:31.170215 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.170119 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34aa7783-4df3-49a2-96f1-a0c801596082" path="/var/lib/kubelet/pods/34aa7783-4df3-49a2-96f1-a0c801596082/volumes" Nov 04 05:45:31.245219 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.245172 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 05:45:31.245438 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.245311 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" podUID="792505fe-6c68-4def-bd32-a41ca2a1fb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 05:45:31.471525 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.471476 5904 generic.go:356] "Generic (PLEG): container finished" podID="ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf" containerID="2b394d317c2caf3274e1eb68628ca93ab3714d9f3a2b65c6657dfcfe1e193eed" exitCode=0 Nov 04 05:45:31.472049 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.471546 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" Nov 04 05:45:31.472049 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.471578 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" event={"ID":"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf","Type":"ContainerDied","Data":"2b394d317c2caf3274e1eb68628ca93ab3714d9f3a2b65c6657dfcfe1e193eed"} Nov 04 05:45:31.472049 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.471613 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns" event={"ID":"ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf","Type":"ContainerDied","Data":"c6483f8c222f538c3181b060e41ee5c1930e6d48f11ac93a9f72923d1d172ecd"} Nov 04 05:45:31.472049 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.471635 5904 scope.go:122] "RemoveContainer" containerID="2b394d317c2caf3274e1eb68628ca93ab3714d9f3a2b65c6657dfcfe1e193eed" Nov 04 05:45:31.479156 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.475264 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" event={"ID":"0b56d92b-5e88-469d-85e0-c7193ac880d9","Type":"ContainerDied","Data":"59800962ace0728abd1d99518b43ff5699dd5d64cb6c448741edd7c933e48b1b"} Nov 04 05:45:31.479711 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.479679 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm" Nov 04 05:45:31.489039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.488378 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" event={"ID":"7c590b80-71fd-4378-bf0d-b296ad006152","Type":"ContainerDied","Data":"cf9c2a882758182c7f50410429a77ff5da4355bd53f50dbad24d08d57824c93e"} Nov 04 05:45:31.489039 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.488539 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" Nov 04 05:45:31.492356 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.492077 5904 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm" podUID="7c590b80-71fd-4378-bf0d-b296ad006152" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded" Nov 04 05:45:31.497029 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.497004 5904 scope.go:122] "RemoveContainer" containerID="2b394d317c2caf3274e1eb68628ca93ab3714d9f3a2b65c6657dfcfe1e193eed" Nov 04 05:45:31.497564 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:45:31.497390 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b394d317c2caf3274e1eb68628ca93ab3714d9f3a2b65c6657dfcfe1e193eed\": container with ID starting with 2b394d317c2caf3274e1eb68628ca93ab3714d9f3a2b65c6657dfcfe1e193eed not found: ID does not exist" containerID="2b394d317c2caf3274e1eb68628ca93ab3714d9f3a2b65c6657dfcfe1e193eed" Nov 04 05:45:31.497564 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.497452 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b394d317c2caf3274e1eb68628ca93ab3714d9f3a2b65c6657dfcfe1e193eed"} err="failed to get container status \"2b394d317c2caf3274e1eb68628ca93ab3714d9f3a2b65c6657dfcfe1e193eed\": rpc error: code = NotFound desc = could not find container \"2b394d317c2caf3274e1eb68628ca93ab3714d9f3a2b65c6657dfcfe1e193eed\": container with ID starting with 2b394d317c2caf3274e1eb68628ca93ab3714d9f3a2b65c6657dfcfe1e193eed not found: ID does not exist" Nov 04 05:45:31.497564 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.497483 5904 scope.go:122] "RemoveContainer" containerID="1976aae86b272633df82be222e7fd881e00ad89576025eb9253a85e68668e9b4" Nov 04 05:45:31.520967 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.520886 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm"] Nov 04 05:45:31.523824 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.523801 5904 scope.go:122] "RemoveContainer" containerID="7797d2187640256c4e277a52a70edbf20ace2aa31ac5c14cc7a5f79ff0b4493d" Nov 04 05:45:31.524741 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.524709 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-27935d/distinct-2-w0-27935d-kserve-7cbbfb45b8-m56tm"] Nov 04 05:45:31.538375 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.538312 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns"] Nov 04 05:45:31.549816 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.549784 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-f910f5/premium-w3-f910f5-kserve-6b7fdc9d7b-zj5ns"] Nov 04 05:45:31.565269 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.565195 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm"] Nov 04 05:45:31.569868 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.569824 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-27935d/distinct-w0-27935d-kserve-64b84bf975-kwqwm"] Nov 04 05:45:31.670588 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.670521 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:31.670883 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.670862 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:31.671657 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:31.671639 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:45:33.168294 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:33.168261 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b56d92b-5e88-469d-85e0-c7193ac880d9" path="/var/lib/kubelet/pods/0b56d92b-5e88-469d-85e0-c7193ac880d9/volumes" Nov 04 05:45:33.168716 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:33.168702 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c590b80-71fd-4378-bf0d-b296ad006152" path="/var/lib/kubelet/pods/7c590b80-71fd-4378-bf0d-b296ad006152/volumes" Nov 04 05:45:33.169073 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:33.169061 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf" path="/var/lib/kubelet/pods/ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf/volumes" Nov 04 05:45:34.047750 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:34.047697 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" podUID="0d9a7888-648f-441c-8f19-281ec23a306e" containerName="payload-processing" containerID="cri-o://9f1437507206c879a9137a008e6e8d8705cacdfbab3d6913fc3fe33e2c68eb67" gracePeriod=25 Nov 04 05:45:39.368895 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:39.368863 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m"] Nov 04 05:45:39.369411 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:39.369110 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" podUID="16013ec4-91d6-4997-ae3f-7545cba44f5c" containerName="maas-api" containerID="cri-o://55ee4dfea074461831feebc85b648430c0d4ebfb75f3ed32483569238cb27e14" gracePeriod=30 Nov 04 05:45:39.595503 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:39.595467 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l"] Nov 04 05:45:39.595740 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:39.595709 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" podUID="a9b1fde1-970f-4864-ac83-a2ce6110cbad" containerName="maas-api" containerID="cri-o://0db01921b4034377093fd224ad39a52a57d87c27543b6b46c8403ef78d76acc0" gracePeriod=30 Nov 04 05:45:39.948473 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:39.948163 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:45:40.012938 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.012906 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:45:40.062476 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.062439 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9b1fde1-970f-4864-ac83-a2ce6110cbad-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b1fde1-970f-4864-ac83-a2ce6110cbad-maas-api-tls\") pod \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\" (UID: \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\") " Nov 04 05:45:40.062638 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.062577 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9b1fde1-970f-4864-ac83-a2ce6110cbad-kube-api-access-k8fpz\" (UniqueName: \"kubernetes.io/projected/a9b1fde1-970f-4864-ac83-a2ce6110cbad-kube-api-access-k8fpz\") pod \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\" (UID: \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\") " Nov 04 05:45:40.062698 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.062635 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9b1fde1-970f-4864-ac83-a2ce6110cbad-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b1fde1-970f-4864-ac83-a2ce6110cbad-openshift-service-ca\") pod \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\" (UID: \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\") " Nov 04 05:45:40.062759 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.062707 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9b1fde1-970f-4864-ac83-a2ce6110cbad-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9b1fde1-970f-4864-ac83-a2ce6110cbad-metrics-certs\") pod \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\" (UID: \"a9b1fde1-970f-4864-ac83-a2ce6110cbad\") " Nov 04 05:45:40.066906 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.065328 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9b1fde1-970f-4864-ac83-a2ce6110cbad-openshift-service-ca" pod "a9b1fde1-970f-4864-ac83-a2ce6110cbad" (UID: "a9b1fde1-970f-4864-ac83-a2ce6110cbad"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:40.066906 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.065399 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9b1fde1-970f-4864-ac83-a2ce6110cbad-maas-api-tls" pod "a9b1fde1-970f-4864-ac83-a2ce6110cbad" (UID: "a9b1fde1-970f-4864-ac83-a2ce6110cbad"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:40.066906 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.065414 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9b1fde1-970f-4864-ac83-a2ce6110cbad-kube-api-access-k8fpz" pod "a9b1fde1-970f-4864-ac83-a2ce6110cbad" (UID: "a9b1fde1-970f-4864-ac83-a2ce6110cbad"). InnerVolumeSpecName "kube-api-access-k8fpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:40.071250 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.067575 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9b1fde1-970f-4864-ac83-a2ce6110cbad-metrics-certs" pod "a9b1fde1-970f-4864-ac83-a2ce6110cbad" (UID: "a9b1fde1-970f-4864-ac83-a2ce6110cbad"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:40.166415 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.166334 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-maas-api-tls\") pod \"16013ec4-91d6-4997-ae3f-7545cba44f5c\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " Nov 04 05:45:40.166560 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.166543 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16013ec4-91d6-4997-ae3f-7545cba44f5c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16013ec4-91d6-4997-ae3f-7545cba44f5c-openshift-service-ca\") pod \"16013ec4-91d6-4997-ae3f-7545cba44f5c\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " Nov 04 05:45:40.166604 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.166590 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-metrics-certs\") pod \"16013ec4-91d6-4997-ae3f-7545cba44f5c\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " Nov 04 05:45:40.166636 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.166622 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16013ec4-91d6-4997-ae3f-7545cba44f5c-kube-api-access-vzk2q\" (UniqueName: \"kubernetes.io/projected/16013ec4-91d6-4997-ae3f-7545cba44f5c-kube-api-access-vzk2q\") pod \"16013ec4-91d6-4997-ae3f-7545cba44f5c\" (UID: \"16013ec4-91d6-4997-ae3f-7545cba44f5c\") " Nov 04 05:45:40.166977 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.166944 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16013ec4-91d6-4997-ae3f-7545cba44f5c-openshift-service-ca" pod "16013ec4-91d6-4997-ae3f-7545cba44f5c" (UID: "16013ec4-91d6-4997-ae3f-7545cba44f5c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:40.167088 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.167048 5904 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b1fde1-970f-4864-ac83-a2ce6110cbad-maas-api-tls\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:40.167088 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.167062 5904 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16013ec4-91d6-4997-ae3f-7545cba44f5c-openshift-service-ca\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:40.167088 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.167073 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8fpz\" (UniqueName: \"kubernetes.io/projected/a9b1fde1-970f-4864-ac83-a2ce6110cbad-kube-api-access-k8fpz\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:40.167088 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.167081 5904 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b1fde1-970f-4864-ac83-a2ce6110cbad-openshift-service-ca\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:40.167355 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.167090 5904 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9b1fde1-970f-4864-ac83-a2ce6110cbad-metrics-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:40.168670 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.168642 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-maas-api-tls" pod "16013ec4-91d6-4997-ae3f-7545cba44f5c" (UID: "16013ec4-91d6-4997-ae3f-7545cba44f5c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:40.169046 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.169031 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-metrics-certs" pod "16013ec4-91d6-4997-ae3f-7545cba44f5c" (UID: "16013ec4-91d6-4997-ae3f-7545cba44f5c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:40.169108 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.169072 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16013ec4-91d6-4997-ae3f-7545cba44f5c-kube-api-access-vzk2q" pod "16013ec4-91d6-4997-ae3f-7545cba44f5c" (UID: "16013ec4-91d6-4997-ae3f-7545cba44f5c"). InnerVolumeSpecName "kube-api-access-vzk2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:40.267842 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.267798 5904 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-maas-api-tls\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:40.267842 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.267827 5904 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16013ec4-91d6-4997-ae3f-7545cba44f5c-metrics-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:40.267842 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.267838 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzk2q\" (UniqueName: \"kubernetes.io/projected/16013ec4-91d6-4997-ae3f-7545cba44f5c-kube-api-access-vzk2q\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:40.548802 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.548710 5904 generic.go:356] "Generic (PLEG): container finished" podID="a9b1fde1-970f-4864-ac83-a2ce6110cbad" containerID="0db01921b4034377093fd224ad39a52a57d87c27543b6b46c8403ef78d76acc0" exitCode=0 Nov 04 05:45:40.548802 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.548779 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" Nov 04 05:45:40.548802 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.548792 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" event={"ID":"a9b1fde1-970f-4864-ac83-a2ce6110cbad","Type":"ContainerDied","Data":"0db01921b4034377093fd224ad39a52a57d87c27543b6b46c8403ef78d76acc0"} Nov 04 05:45:40.549436 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.548848 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" event={"ID":"a9b1fde1-970f-4864-ac83-a2ce6110cbad","Type":"ContainerDied","Data":"72f0d2442ee3ddbc9fec10f3ac259baaa5098f837d24b9bb19bbece9191ca6aa"} Nov 04 05:45:40.549436 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.548866 5904 scope.go:122] "RemoveContainer" containerID="0db01921b4034377093fd224ad39a52a57d87c27543b6b46c8403ef78d76acc0" Nov 04 05:45:40.550281 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.550254 5904 generic.go:356] "Generic (PLEG): container finished" podID="16013ec4-91d6-4997-ae3f-7545cba44f5c" containerID="55ee4dfea074461831feebc85b648430c0d4ebfb75f3ed32483569238cb27e14" exitCode=0 Nov 04 05:45:40.550374 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.550285 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" event={"ID":"16013ec4-91d6-4997-ae3f-7545cba44f5c","Type":"ContainerDied","Data":"55ee4dfea074461831feebc85b648430c0d4ebfb75f3ed32483569238cb27e14"} Nov 04 05:45:40.550374 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.550332 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" Nov 04 05:45:40.550374 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.550342 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m" event={"ID":"16013ec4-91d6-4997-ae3f-7545cba44f5c","Type":"ContainerDied","Data":"3165e6b7b344c4802823e212485c0d508c8736c7e9482f11551ce2c5d7491c18"} Nov 04 05:45:40.560012 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.559996 5904 scope.go:122] "RemoveContainer" containerID="0db01921b4034377093fd224ad39a52a57d87c27543b6b46c8403ef78d76acc0" Nov 04 05:45:40.560338 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:45:40.560313 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0db01921b4034377093fd224ad39a52a57d87c27543b6b46c8403ef78d76acc0\": container with ID starting with 0db01921b4034377093fd224ad39a52a57d87c27543b6b46c8403ef78d76acc0 not found: ID does not exist" containerID="0db01921b4034377093fd224ad39a52a57d87c27543b6b46c8403ef78d76acc0" Nov 04 05:45:40.560435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.560348 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0db01921b4034377093fd224ad39a52a57d87c27543b6b46c8403ef78d76acc0"} err="failed to get container status \"0db01921b4034377093fd224ad39a52a57d87c27543b6b46c8403ef78d76acc0\": rpc error: code = NotFound desc = could not find container \"0db01921b4034377093fd224ad39a52a57d87c27543b6b46c8403ef78d76acc0\": container with ID starting with 0db01921b4034377093fd224ad39a52a57d87c27543b6b46c8403ef78d76acc0 not found: ID does not exist" Nov 04 05:45:40.560435 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.560366 5904 scope.go:122] "RemoveContainer" containerID="55ee4dfea074461831feebc85b648430c0d4ebfb75f3ed32483569238cb27e14" Nov 04 05:45:40.568993 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.568975 5904 scope.go:122] "RemoveContainer" containerID="55ee4dfea074461831feebc85b648430c0d4ebfb75f3ed32483569238cb27e14" Nov 04 05:45:40.569283 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:45:40.569264 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55ee4dfea074461831feebc85b648430c0d4ebfb75f3ed32483569238cb27e14\": container with ID starting with 55ee4dfea074461831feebc85b648430c0d4ebfb75f3ed32483569238cb27e14 not found: ID does not exist" containerID="55ee4dfea074461831feebc85b648430c0d4ebfb75f3ed32483569238cb27e14" Nov 04 05:45:40.569331 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.569291 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55ee4dfea074461831feebc85b648430c0d4ebfb75f3ed32483569238cb27e14"} err="failed to get container status \"55ee4dfea074461831feebc85b648430c0d4ebfb75f3ed32483569238cb27e14\": rpc error: code = NotFound desc = could not find container \"55ee4dfea074461831feebc85b648430c0d4ebfb75f3ed32483569238cb27e14\": container with ID starting with 55ee4dfea074461831feebc85b648430c0d4ebfb75f3ed32483569238cb27e14 not found: ID does not exist" Nov 04 05:45:40.574893 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.574872 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l"] Nov 04 05:45:40.578250 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.578215 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l"] Nov 04 05:45:40.587804 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.587782 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m"] Nov 04 05:45:40.589749 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:40.589730 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-27935d-54dd4977df-w2n5m"] Nov 04 05:45:41.169178 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:41.169147 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16013ec4-91d6-4997-ae3f-7545cba44f5c" path="/var/lib/kubelet/pods/16013ec4-91d6-4997-ae3f-7545cba44f5c/volumes" Nov 04 05:45:41.169606 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:41.169592 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9b1fde1-970f-4864-ac83-a2ce6110cbad" path="/var/lib/kubelet/pods/a9b1fde1-970f-4864-ac83-a2ce6110cbad/volumes" Nov 04 05:45:42.782818 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:42.782779 5904 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-f910f5-86f6b57569-lrf7l" podUID="a9b1fde1-970f-4864-ac83-a2ce6110cbad" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.44:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 04 05:45:42.811173 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:42.811137 5904 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" podUID="0d9a7888-648f-441c-8f19-281ec23a306e" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:45:45.487266 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:45.487205 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g"] Nov 04 05:45:45.488140 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:45.488115 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" podUID="2e0612d3-74c4-4d7c-9306-13ed78cf1b47" containerName="istio-proxy" containerID="cri-o://8ed29c738a8b86394df6c2c72b2246af556e54f2394db250e24b367417602864" gracePeriod=30 Nov 04 05:45:51.247048 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.247015 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:51.320094 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.320064 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istiod-ca-cert\") pod \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " Nov 04 05:45:51.320279 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.320142 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-envoy\") pod \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " Nov 04 05:45:51.320279 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.320161 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-data\") pod \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " Nov 04 05:45:51.320279 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.320185 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-token\" (UniqueName: \"kubernetes.io/projected/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-token\") pod \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " Nov 04 05:45:51.320425 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.320298 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-workload-socket\") pod \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " Nov 04 05:45:51.320425 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.320335 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-kube-api-access-lt2lq\" (UniqueName: \"kubernetes.io/projected/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-kube-api-access-lt2lq\") pod \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " Nov 04 05:45:51.320522 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.320422 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-credential-socket\") pod \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " Nov 04 05:45:51.320522 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.320451 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-workload-certs\") pod \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " Nov 04 05:45:51.320522 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.320481 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-podinfo\") pod \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\" (UID: \"2e0612d3-74c4-4d7c-9306-13ed78cf1b47\") " Nov 04 05:45:51.320522 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.320494 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-data" pod "2e0612d3-74c4-4d7c-9306-13ed78cf1b47" (UID: "2e0612d3-74c4-4d7c-9306-13ed78cf1b47"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:51.320723 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.320526 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istiod-ca-cert" pod "2e0612d3-74c4-4d7c-9306-13ed78cf1b47" (UID: "2e0612d3-74c4-4d7c-9306-13ed78cf1b47"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:51.320723 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.320666 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-workload-socket" pod "2e0612d3-74c4-4d7c-9306-13ed78cf1b47" (UID: "2e0612d3-74c4-4d7c-9306-13ed78cf1b47"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:51.320817 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.320718 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-credential-socket" pod "2e0612d3-74c4-4d7c-9306-13ed78cf1b47" (UID: "2e0612d3-74c4-4d7c-9306-13ed78cf1b47"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:51.320914 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.320890 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-workload-certs" pod "2e0612d3-74c4-4d7c-9306-13ed78cf1b47" (UID: "2e0612d3-74c4-4d7c-9306-13ed78cf1b47"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:51.321114 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.320958 5904 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istiod-ca-cert\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:51.321215 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.321128 5904 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-data\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:51.321215 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.321145 5904 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-workload-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:51.321215 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.321161 5904 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-credential-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:51.322441 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.322419 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-envoy" pod "2e0612d3-74c4-4d7c-9306-13ed78cf1b47" (UID: "2e0612d3-74c4-4d7c-9306-13ed78cf1b47"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:51.322597 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.322577 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-podinfo" pod "2e0612d3-74c4-4d7c-9306-13ed78cf1b47" (UID: "2e0612d3-74c4-4d7c-9306-13ed78cf1b47"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:45:51.322916 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.322893 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-kube-api-access-lt2lq" pod "2e0612d3-74c4-4d7c-9306-13ed78cf1b47" (UID: "2e0612d3-74c4-4d7c-9306-13ed78cf1b47"). InnerVolumeSpecName "kube-api-access-lt2lq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:51.322979 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.322915 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-token" pod "2e0612d3-74c4-4d7c-9306-13ed78cf1b47" (UID: "2e0612d3-74c4-4d7c-9306-13ed78cf1b47"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:51.422509 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.422477 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt2lq\" (UniqueName: \"kubernetes.io/projected/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-kube-api-access-lt2lq\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:51.422509 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.422508 5904 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-workload-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:51.422509 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.422518 5904 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-podinfo\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:51.422727 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.422529 5904 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-envoy\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:51.422727 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.422538 5904 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e0612d3-74c4-4d7c-9306-13ed78cf1b47-istio-token\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:51.437428 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437395 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg"] Nov 04 05:45:51.437836 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437824 5904 cpu_manager.go:414] "containerMap: removing container" podUID="a9b1fde1-970f-4864-ac83-a2ce6110cbad" containerName="maas-api" Nov 04 05:45:51.437885 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437837 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9b1fde1-970f-4864-ac83-a2ce6110cbad" containerName="maas-api" Nov 04 05:45:51.437885 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437846 5904 cpu_manager.go:414] "containerMap: removing container" podUID="0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec" containerName="main" Nov 04 05:45:51.437885 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437852 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec" containerName="main" Nov 04 05:45:51.437885 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437869 5904 cpu_manager.go:414] "containerMap: removing container" podUID="e250af86-5b3a-4ef2-879c-022bb361c11e" containerName="istio-proxy" Nov 04 05:45:51.437885 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437875 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e250af86-5b3a-4ef2-879c-022bb361c11e" containerName="istio-proxy" Nov 04 05:45:51.437885 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437885 5904 cpu_manager.go:414] "containerMap: removing container" podUID="7c590b80-71fd-4378-bf0d-b296ad006152" containerName="main" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437890 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c590b80-71fd-4378-bf0d-b296ad006152" containerName="main" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437898 5904 cpu_manager.go:414] "containerMap: removing container" podUID="106302a4-351e-43c6-a5c5-a032a085ccc4" containerName="main" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437902 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="106302a4-351e-43c6-a5c5-a032a085ccc4" containerName="main" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437908 5904 cpu_manager.go:414] "containerMap: removing container" podUID="0b56d92b-5e88-469d-85e0-c7193ac880d9" containerName="main" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437913 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b56d92b-5e88-469d-85e0-c7193ac880d9" containerName="main" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437920 5904 cpu_manager.go:414] "containerMap: removing container" podUID="ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf" containerName="main" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437924 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf" containerName="main" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437934 5904 cpu_manager.go:414] "containerMap: removing container" podUID="16013ec4-91d6-4997-ae3f-7545cba44f5c" containerName="maas-api" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437939 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16013ec4-91d6-4997-ae3f-7545cba44f5c" containerName="maas-api" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437948 5904 cpu_manager.go:414] "containerMap: removing container" podUID="2e0612d3-74c4-4d7c-9306-13ed78cf1b47" containerName="istio-proxy" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437952 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e0612d3-74c4-4d7c-9306-13ed78cf1b47" containerName="istio-proxy" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437963 5904 cpu_manager.go:414] "containerMap: removing container" podUID="34aa7783-4df3-49a2-96f1-a0c801596082" containerName="maas-api" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.437968 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34aa7783-4df3-49a2-96f1-a0c801596082" containerName="maas-api" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.438028 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="34aa7783-4df3-49a2-96f1-a0c801596082" containerName="maas-api" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.438038 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9b1fde1-970f-4864-ac83-a2ce6110cbad" containerName="maas-api" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.438044 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ebf1b07-2f78-41f9-9e9f-0b93e4c6e7ec" containerName="main" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.438052 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="e250af86-5b3a-4ef2-879c-022bb361c11e" containerName="istio-proxy" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.438058 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c590b80-71fd-4378-bf0d-b296ad006152" containerName="main" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.438066 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="106302a4-351e-43c6-a5c5-a032a085ccc4" containerName="main" Nov 04 05:45:51.438064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.438072 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b56d92b-5e88-469d-85e0-c7193ac880d9" containerName="main" Nov 04 05:45:51.438967 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.438078 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec3a08cf-c3a9-4f90-b6f9-9ce1a8f0cccf" containerName="main" Nov 04 05:45:51.438967 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.438084 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="16013ec4-91d6-4997-ae3f-7545cba44f5c" containerName="maas-api" Nov 04 05:45:51.438967 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.438090 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e0612d3-74c4-4d7c-9306-13ed78cf1b47" containerName="istio-proxy" Nov 04 05:45:51.442802 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.442779 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.456125 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.455377 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg"] Nov 04 05:45:51.523171 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.523135 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f4fab8f-362d-4834-bd81-a139841c6bcb-istiod-ca-cert\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.523416 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.523211 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-credential-socket\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.523416 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.523349 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwbln\" (UniqueName: \"kubernetes.io/projected/1f4fab8f-362d-4834-bd81-a139841c6bcb-kube-api-access-nwbln\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.523416 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.523401 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-podinfo\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.523528 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.523452 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-envoy\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.523528 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.523513 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-data\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.523603 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.523587 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-workload-socket\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.523680 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.523659 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-token\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.523728 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.523712 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-workload-certs\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.628101 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.625877 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-token\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.628101 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.625947 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-workload-certs\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.628101 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.626033 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f4fab8f-362d-4834-bd81-a139841c6bcb-istiod-ca-cert\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.628101 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.626090 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-credential-socket\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.628101 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.626147 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwbln\" (UniqueName: \"kubernetes.io/projected/1f4fab8f-362d-4834-bd81-a139841c6bcb-kube-api-access-nwbln\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.628101 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.626183 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-podinfo\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.628101 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.626261 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-envoy\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.628101 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.626290 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-data\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.628101 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.626352 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-workload-socket\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.628101 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.626853 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-workload-socket\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.628101 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.627088 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-workload-certs\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.628811 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.628439 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f4fab8f-362d-4834-bd81-a139841c6bcb-istiod-ca-cert\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.631393 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.631334 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-envoy\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.631548 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.631441 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-data\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.633247 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.631979 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-credential-socket\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.633247 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.632167 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-podinfo\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.639402 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.638976 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-token\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.643038 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.643015 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwbln\" (UniqueName: \"kubernetes.io/projected/1f4fab8f-362d-4834-bd81-a139841c6bcb-kube-api-access-nwbln\") pod \"e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.674217 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.674178 5904 generic.go:356] "Generic (PLEG): container finished" podID="2e0612d3-74c4-4d7c-9306-13ed78cf1b47" containerID="8ed29c738a8b86394df6c2c72b2246af556e54f2394db250e24b367417602864" exitCode=0 Nov 04 05:45:51.674350 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.674256 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" event={"ID":"2e0612d3-74c4-4d7c-9306-13ed78cf1b47","Type":"ContainerDied","Data":"8ed29c738a8b86394df6c2c72b2246af556e54f2394db250e24b367417602864"} Nov 04 05:45:51.674350 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.674332 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" event={"ID":"2e0612d3-74c4-4d7c-9306-13ed78cf1b47","Type":"ContainerDied","Data":"5f7975af176b7e7ae14204c95f274156380e02a00edb6fcfaae0aa1204ac6380"} Nov 04 05:45:51.674350 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.674261 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g" Nov 04 05:45:51.674458 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.674359 5904 scope.go:122] "RemoveContainer" containerID="8ed29c738a8b86394df6c2c72b2246af556e54f2394db250e24b367417602864" Nov 04 05:45:51.678791 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.678768 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg"] Nov 04 05:45:51.684142 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.684016 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.686341 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.686323 5904 scope.go:122] "RemoveContainer" containerID="8ed29c738a8b86394df6c2c72b2246af556e54f2394db250e24b367417602864" Nov 04 05:45:51.686632 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:45:51.686613 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ed29c738a8b86394df6c2c72b2246af556e54f2394db250e24b367417602864\": container with ID starting with 8ed29c738a8b86394df6c2c72b2246af556e54f2394db250e24b367417602864 not found: ID does not exist" containerID="8ed29c738a8b86394df6c2c72b2246af556e54f2394db250e24b367417602864" Nov 04 05:45:51.686700 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.686640 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ed29c738a8b86394df6c2c72b2246af556e54f2394db250e24b367417602864"} err="failed to get container status \"8ed29c738a8b86394df6c2c72b2246af556e54f2394db250e24b367417602864\": rpc error: code = NotFound desc = could not find container \"8ed29c738a8b86394df6c2c72b2246af556e54f2394db250e24b367417602864\": container with ID starting with 8ed29c738a8b86394df6c2c72b2246af556e54f2394db250e24b367417602864 not found: ID does not exist" Nov 04 05:45:51.696654 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.696632 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg"] Nov 04 05:45:51.714600 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.714572 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk"] Nov 04 05:45:51.718871 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.718854 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.727387 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.727094 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istiod-ca-cert\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.727387 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.727166 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-data\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.727387 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.727251 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-token\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.727387 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.727285 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-workload-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.727387 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.727344 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-credential-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.728330 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.728300 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-envoy\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.728439 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.728345 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-workload-certs\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.728439 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.728395 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-podinfo\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.728439 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.728426 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsh2n\" (UniqueName: \"kubernetes.io/projected/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-kube-api-access-qsh2n\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.729662 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.729640 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk"] Nov 04 05:45:51.732130 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.732059 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g"] Nov 04 05:45:51.742364 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.742338 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-2b14d0-openshift-default-57ff65bf59-75k5g"] Nov 04 05:45:51.760290 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.760262 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:51.829839 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.829807 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-token\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.829959 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.829864 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-token\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.830000 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.829961 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-workload-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.830048 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.830031 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-credential-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.830080 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.830067 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-podinfo\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.830119 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.830100 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-workload-certs\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.830156 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.830129 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-credential-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.830195 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.830167 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-envoy\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.830255 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.830208 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-workload-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.830300 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.830256 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-envoy\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.830461 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.830442 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-workload-certs\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.831077 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.830747 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-podinfo\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.831077 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.830772 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-workload-certs\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.831077 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.830998 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-credential-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.831077 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.830775 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-data\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.831077 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.831045 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-workload-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.831077 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.831065 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsh2n\" (UniqueName: \"kubernetes.io/projected/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-kube-api-access-qsh2n\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.831399 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.831099 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istiod-ca-cert\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.831399 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.831123 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqptk\" (UniqueName: \"kubernetes.io/projected/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-kube-api-access-lqptk\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.831399 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.831179 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istiod-ca-cert\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.831399 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.831265 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-data\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.831674 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.831653 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-data\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.832431 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.832410 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istiod-ca-cert\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.833475 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.833454 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-envoy\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.833637 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.833619 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-podinfo\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.838587 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.838565 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-token\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.838684 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.838584 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsh2n\" (UniqueName: \"kubernetes.io/projected/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-kube-api-access-qsh2n\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:51.932688 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.932611 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-token\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.932688 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.932668 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-podinfo\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.932911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.932693 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-workload-certs\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.932911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.932711 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-credential-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.932911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.932730 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-envoy\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.932911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.932763 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-workload-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.932911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.932815 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-data\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.932911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.932847 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istiod-ca-cert\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.932911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.932873 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqptk\" (UniqueName: \"kubernetes.io/projected/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-kube-api-access-lqptk\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.933302 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.933163 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-workload-certs\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.933302 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.933222 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-workload-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.933418 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.933312 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-data\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.933559 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.933540 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-credential-socket\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.933636 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.933605 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istiod-ca-cert\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.934995 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.934975 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-envoy\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.935275 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.935258 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-podinfo\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.939834 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.939810 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-token\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:51.939963 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:51.939947 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqptk\" (UniqueName: \"kubernetes.io/projected/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-kube-api-access-lqptk\") pod \"e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:52.008664 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.004577 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:52.036929 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.036894 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:52.052701 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.052678 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg"] Nov 04 05:45:52.054943 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:45:52.054916 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f4fab8f_362d_4834_bd81_a139841c6bcb.slice/crio-3b896263928aaa02d83a695909e0873b4f144d449896fc52b8b2f788cfb0dbca WatchSource:0}: Error finding container 3b896263928aaa02d83a695909e0873b4f144d449896fc52b8b2f788cfb0dbca: Status 404 returned error can't find the container with id 3b896263928aaa02d83a695909e0873b4f144d449896fc52b8b2f788cfb0dbca Nov 04 05:45:52.057969 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.057935 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:52.058080 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.058019 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:52.058080 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.058058 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:52.302610 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.302578 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg"] Nov 04 05:45:52.303613 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:45:52.303581 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc81ca4c7_8ce0_4591_bf5b_ef6f6ff5a8d1.slice/crio-fc51d7e589c2eec434313aff359a08c6bbc1a1f32fc63ada12500f3162eef20f WatchSource:0}: Error finding container fc51d7e589c2eec434313aff359a08c6bbc1a1f32fc63ada12500f3162eef20f: Status 404 returned error can't find the container with id fc51d7e589c2eec434313aff359a08c6bbc1a1f32fc63ada12500f3162eef20f Nov 04 05:45:52.320919 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.320882 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:52.321152 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.321121 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:52.321301 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.321189 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:52.359118 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.359087 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk"] Nov 04 05:45:52.372429 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.372390 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:52.372533 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.372480 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:52.372533 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.372522 5904 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:52.685306 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.684355 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" event={"ID":"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42","Type":"ContainerStarted","Data":"14dbca9f8afa7e052892723c18f4180fb109345487d76750076c6ccc711ae626"} Nov 04 05:45:52.685306 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.684422 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" event={"ID":"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42","Type":"ContainerStarted","Data":"87b672981a3bd5a1ab82466a2c377e157a957d4caeebf426f5be915537663007"} Nov 04 05:45:52.687418 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.687391 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" event={"ID":"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1","Type":"ContainerStarted","Data":"1dbb4f310e1a36a5344e20bb1e0536f1b611e42022b1d5095d84e32e4274989b"} Nov 04 05:45:52.687655 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.687624 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" event={"ID":"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1","Type":"ContainerStarted","Data":"fc51d7e589c2eec434313aff359a08c6bbc1a1f32fc63ada12500f3162eef20f"} Nov 04 05:45:52.691457 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.691431 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" event={"ID":"1f4fab8f-362d-4834-bd81-a139841c6bcb","Type":"ContainerStarted","Data":"3cc6637e2eaf10b27b6df0bdf113ecbd01b947c368f788bb49637a410755e3a9"} Nov 04 05:45:52.691557 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.691484 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" event={"ID":"1f4fab8f-362d-4834-bd81-a139841c6bcb","Type":"ContainerStarted","Data":"3b896263928aaa02d83a695909e0873b4f144d449896fc52b8b2f788cfb0dbca"} Nov 04 05:45:52.706197 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.705882 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" podStartSLOduration=1.705865233 podStartE2EDuration="1.705865233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:52.704384893 +0000 UTC m=+999.661969844" watchObservedRunningTime="2026-11-04 05:45:52.705865233 +0000 UTC m=+999.663450184" Nov 04 05:45:52.745604 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.745544 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" podStartSLOduration=1.7455250759999998 podStartE2EDuration="1.745525076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:52.724067613 +0000 UTC m=+999.681652564" watchObservedRunningTime="2026-11-04 05:45:52.745525076 +0000 UTC m=+999.703110027" Nov 04 05:45:52.745994 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.745952 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" podStartSLOduration=1.74593844 podStartE2EDuration="1.74593844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:52.744492428 +0000 UTC m=+999.702077378" watchObservedRunningTime="2026-11-04 05:45:52.74593844 +0000 UTC m=+999.703523402" Nov 04 05:45:52.760832 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.760797 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:45:52.811353 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:52.811310 5904 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" podUID="0d9a7888-648f-441c-8f19-281ec23a306e" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:45:53.018371 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:53.008670 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:45:53.018371 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:53.017705 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Nov 04 05:45:53.018371 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:53.017762 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" podUID="c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Nov 04 05:45:53.038282 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:53.037758 5904 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:45:53.038864 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:53.038815 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:53.039080 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:53.038885 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" podUID="9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Nov 04 05:45:53.337264 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:53.334786 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e0612d3-74c4-4d7c-9306-13ed78cf1b47" path="/var/lib/kubelet/pods/2e0612d3-74c4-4d7c-9306-13ed78cf1b47/volumes" Nov 04 05:45:53.762787 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:53.762436 5904 patch_prober.go:29] interesting pod/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Nov 04 05:45:53.762787 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:53.762489 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" podUID="1f4fab8f-362d-4834-bd81-a139841c6bcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Nov 04 05:45:54.013807 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:54.013716 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Nov 04 05:45:54.014089 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:54.014049 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" podUID="c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Nov 04 05:45:54.038653 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:54.038614 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:54.038900 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:54.038869 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" podUID="9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Nov 04 05:45:54.859279 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:54.835275 5904 patch_prober.go:29] interesting pod/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Nov 04 05:45:54.878307 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:54.876718 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" podUID="1f4fab8f-362d-4834-bd81-a139841c6bcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Nov 04 05:45:54.878307 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:54.848808 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" event={"ID":"0d9a7888-648f-441c-8f19-281ec23a306e","Type":"ContainerDied","Data":"9f1437507206c879a9137a008e6e8d8705cacdfbab3d6913fc3fe33e2c68eb67"} Nov 04 05:45:54.878307 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:54.848779 5904 generic.go:356] "Generic (PLEG): container finished" podID="0d9a7888-648f-441c-8f19-281ec23a306e" containerID="9f1437507206c879a9137a008e6e8d8705cacdfbab3d6913fc3fe33e2c68eb67" exitCode=0 Nov 04 05:45:55.024263 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.009324 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Nov 04 05:45:55.024263 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.009380 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" podUID="c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Nov 04 05:45:55.052282 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.050454 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:55.052282 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.050514 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" podUID="9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Nov 04 05:45:55.214574 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.214487 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" Nov 04 05:45:55.374722 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.374679 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d9a7888-648f-441c-8f19-281ec23a306e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d9a7888-648f-441c-8f19-281ec23a306e-plugins-config-volume\") pod \"0d9a7888-648f-441c-8f19-281ec23a306e\" (UID: \"0d9a7888-648f-441c-8f19-281ec23a306e\") " Nov 04 05:45:55.375082 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.375060 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d9a7888-648f-441c-8f19-281ec23a306e-kube-api-access-mr65x\" (UniqueName: \"kubernetes.io/projected/0d9a7888-648f-441c-8f19-281ec23a306e-kube-api-access-mr65x\") pod \"0d9a7888-648f-441c-8f19-281ec23a306e\" (UID: \"0d9a7888-648f-441c-8f19-281ec23a306e\") " Nov 04 05:45:55.383109 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.383077 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d9a7888-648f-441c-8f19-281ec23a306e-plugins-config-volume" pod "0d9a7888-648f-441c-8f19-281ec23a306e" (UID: "0d9a7888-648f-441c-8f19-281ec23a306e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:55.428268 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.428185 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d9a7888-648f-441c-8f19-281ec23a306e-kube-api-access-mr65x" pod "0d9a7888-648f-441c-8f19-281ec23a306e" (UID: "0d9a7888-648f-441c-8f19-281ec23a306e"). InnerVolumeSpecName "kube-api-access-mr65x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:55.492740 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.480259 5904 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d9a7888-648f-441c-8f19-281ec23a306e-plugins-config-volume\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:55.492740 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.480301 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mr65x\" (UniqueName: \"kubernetes.io/projected/0d9a7888-648f-441c-8f19-281ec23a306e-kube-api-access-mr65x\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:55.782307 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.765485 5904 patch_prober.go:29] interesting pod/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Nov 04 05:45:55.782307 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.765545 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" podUID="1f4fab8f-362d-4834-bd81-a139841c6bcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Nov 04 05:45:55.930282 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.926524 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" event={"ID":"0d9a7888-648f-441c-8f19-281ec23a306e","Type":"ContainerDied","Data":"ca3a8bc51c4566e6d9601ed3d5b161d646ee9912952b199e69bea4006b63719e"} Nov 04 05:45:55.930282 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.926599 5904 scope.go:122] "RemoveContainer" containerID="9f1437507206c879a9137a008e6e8d8705cacdfbab3d6913fc3fe33e2c68eb67" Nov 04 05:45:55.930282 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.926739 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj" Nov 04 05:45:55.998352 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:55.998280 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj"] Nov 04 05:45:56.014102 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:56.012781 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Nov 04 05:45:56.014102 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:56.012836 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" podUID="c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Nov 04 05:45:56.015467 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:56.015399 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-9d808e-877db75d8-5nlfj"] Nov 04 05:45:56.041116 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:56.040648 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:56.041116 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:56.040711 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" podUID="9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Nov 04 05:45:56.762247 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:56.761582 5904 patch_prober.go:29] interesting pod/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Nov 04 05:45:56.762247 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:56.761640 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" podUID="1f4fab8f-362d-4834-bd81-a139841c6bcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Nov 04 05:45:57.008114 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:57.007674 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Nov 04 05:45:57.008114 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:57.007737 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" podUID="c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Nov 04 05:45:57.037828 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:57.037734 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:57.038052 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:57.038024 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" podUID="9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Nov 04 05:45:57.225138 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:57.224935 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d9a7888-648f-441c-8f19-281ec23a306e" path="/var/lib/kubelet/pods/0d9a7888-648f-441c-8f19-281ec23a306e/volumes" Nov 04 05:45:57.225726 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:45:57.225696 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:45:57.773252 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:57.754791 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw"] Nov 04 05:45:57.773252 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:57.755162 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" podUID="e2b7b8df-514a-43c7-b0a9-920332da4dc6" containerName="istio-proxy" containerID="cri-o://58c14dd957c5241fdd00d493cfcf99606fce04f839e6115972d633ab3f2dfd82" gracePeriod=30 Nov 04 05:45:57.776807 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:57.776768 5904 patch_prober.go:29] interesting pod/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Nov 04 05:45:57.776941 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:57.776849 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" podUID="1f4fab8f-362d-4834-bd81-a139841c6bcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Nov 04 05:45:57.833327 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:57.832368 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg"] Nov 04 05:45:57.833327 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:57.832686 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" podUID="c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" containerName="istio-proxy" containerID="cri-o://1dbb4f310e1a36a5344e20bb1e0536f1b611e42022b1d5095d84e32e4274989b" gracePeriod=30 Nov 04 05:45:58.027429 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:58.027338 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk"] Nov 04 05:45:58.032245 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:58.028178 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" podUID="9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" containerName="istio-proxy" containerID="cri-o://14dbca9f8afa7e052892723c18f4180fb109345487d76750076c6ccc711ae626" gracePeriod=30 Nov 04 05:45:58.243499 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:58.243465 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw"] Nov 04 05:45:58.243979 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:58.243930 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" podUID="353f0a70-20f0-4878-b7fa-20221f62ebc4" containerName="istio-proxy" containerID="cri-o://f3a472abe60d6635427e9e78bce3afe26e220060e57ed1034439baeeb84b8406" gracePeriod=30 Nov 04 05:45:58.291748 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:58.291656 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4"] Nov 04 05:45:58.292213 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:58.292182 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" podUID="792505fe-6c68-4def-bd32-a41ca2a1fb34" containerName="istio-proxy" containerID="cri-o://a37b8db06f38ce129bd02e7746546b61a65a249afca3ea1ccd1eb9f1b62e293f" gracePeriod=30 Nov 04 05:45:58.772252 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:58.768589 5904 patch_prober.go:29] interesting pod/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Nov 04 05:45:58.772252 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:58.768661 5904 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" podUID="1f4fab8f-362d-4834-bd81-a139841c6bcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Nov 04 05:45:58.990380 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:58.981328 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg"] Nov 04 05:45:58.990380 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:58.981594 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" podUID="1f4fab8f-362d-4834-bd81-a139841c6bcb" containerName="istio-proxy" containerID="cri-o://3cc6637e2eaf10b27b6df0bdf113ecbd01b947c368f788bb49637a410755e3a9" gracePeriod=30 Nov 04 05:45:59.134029 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:59.111376 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p"] Nov 04 05:45:59.134029 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:59.111806 5904 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" podUID="0b00d56d-9587-42b9-81a1-1d3cb9787842" containerName="istio-proxy" containerID="cri-o://8fff9c9f288cbcbd0000c74ddd2bc4f8b9a1ca332d2390ecd4e97ecc60838467" gracePeriod=30 Nov 04 05:45:59.695723 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:59.695648 5904 patch_prober.go:29] interesting pod/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:45:59.695723 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:59.695717 5904 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" podUID="0b00d56d-9587-42b9-81a1-1d3cb9787842" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:45:59.945462 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:59.945390 5904 patch_prober.go:29] interesting pod/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:45:59.945462 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:45:59.945452 5904 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" podUID="e2b7b8df-514a-43c7-b0a9-920332da4dc6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:46:00.292084 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:00.292050 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:46:00.292500 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:00.292108 5904 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" podUID="353f0a70-20f0-4878-b7fa-20221f62ebc4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:46:01.673438 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:01.673405 5904 patch_prober.go:29] interesting pod/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:46:01.673807 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:01.673456 5904 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" podUID="792505fe-6c68-4def-bd32-a41ca2a1fb34" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:46:03.291703 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.291677 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:46:03.352997 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.352971 5904 generic.go:356] "Generic (PLEG): container finished" podID="e2b7b8df-514a-43c7-b0a9-920332da4dc6" containerID="58c14dd957c5241fdd00d493cfcf99606fce04f839e6115972d633ab3f2dfd82" exitCode=0 Nov 04 05:46:03.353140 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.353013 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" Nov 04 05:46:03.353140 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.353056 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" event={"ID":"e2b7b8df-514a-43c7-b0a9-920332da4dc6","Type":"ContainerDied","Data":"58c14dd957c5241fdd00d493cfcf99606fce04f839e6115972d633ab3f2dfd82"} Nov 04 05:46:03.353140 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.353109 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw" event={"ID":"e2b7b8df-514a-43c7-b0a9-920332da4dc6","Type":"ContainerDied","Data":"f8c4298971fa917a482b7889cc4c3686809c4e6cb96960656eaf09567ef99f2c"} Nov 04 05:46:03.353140 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.353140 5904 scope.go:122] "RemoveContainer" containerID="58c14dd957c5241fdd00d493cfcf99606fce04f839e6115972d633ab3f2dfd82" Nov 04 05:46:03.355848 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.355197 5904 generic.go:356] "Generic (PLEG): container finished" podID="353f0a70-20f0-4878-b7fa-20221f62ebc4" containerID="f3a472abe60d6635427e9e78bce3afe26e220060e57ed1034439baeeb84b8406" exitCode=0 Nov 04 05:46:03.355848 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.355263 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" event={"ID":"353f0a70-20f0-4878-b7fa-20221f62ebc4","Type":"ContainerDied","Data":"f3a472abe60d6635427e9e78bce3afe26e220060e57ed1034439baeeb84b8406"} Nov 04 05:46:03.393181 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.393110 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-data\") pod \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " Nov 04 05:46:03.393181 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.393169 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-workload-socket\") pod \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " Nov 04 05:46:03.393364 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.393198 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-podinfo\") pod \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " Nov 04 05:46:03.393364 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.393254 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-token\" (UniqueName: \"kubernetes.io/projected/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-token\") pod \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " Nov 04 05:46:03.393364 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.393285 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-envoy\") pod \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " Nov 04 05:46:03.393364 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.393329 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-credential-socket\") pod \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " Nov 04 05:46:03.393486 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.393367 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istiod-ca-cert\") pod \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " Nov 04 05:46:03.393486 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.393392 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-workload-certs\") pod \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " Nov 04 05:46:03.393486 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.393417 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2b7b8df-514a-43c7-b0a9-920332da4dc6-kube-api-access-9tlrn\" (UniqueName: \"kubernetes.io/projected/e2b7b8df-514a-43c7-b0a9-920332da4dc6-kube-api-access-9tlrn\") pod \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\" (UID: \"e2b7b8df-514a-43c7-b0a9-920332da4dc6\") " Nov 04 05:46:03.393957 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.393886 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-workload-socket" pod "e2b7b8df-514a-43c7-b0a9-920332da4dc6" (UID: "e2b7b8df-514a-43c7-b0a9-920332da4dc6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:03.394074 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.393977 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-workload-certs" pod "e2b7b8df-514a-43c7-b0a9-920332da4dc6" (UID: "e2b7b8df-514a-43c7-b0a9-920332da4dc6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:03.394151 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.394069 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-credential-socket" pod "e2b7b8df-514a-43c7-b0a9-920332da4dc6" (UID: "e2b7b8df-514a-43c7-b0a9-920332da4dc6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:03.394151 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.394079 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-data" pod "e2b7b8df-514a-43c7-b0a9-920332da4dc6" (UID: "e2b7b8df-514a-43c7-b0a9-920332da4dc6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:03.394338 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.394310 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istiod-ca-cert" pod "e2b7b8df-514a-43c7-b0a9-920332da4dc6" (UID: "e2b7b8df-514a-43c7-b0a9-920332da4dc6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:03.396007 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.395983 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2b7b8df-514a-43c7-b0a9-920332da4dc6-kube-api-access-9tlrn" pod "e2b7b8df-514a-43c7-b0a9-920332da4dc6" (UID: "e2b7b8df-514a-43c7-b0a9-920332da4dc6"). InnerVolumeSpecName "kube-api-access-9tlrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:03.396217 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.396198 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-token" pod "e2b7b8df-514a-43c7-b0a9-920332da4dc6" (UID: "e2b7b8df-514a-43c7-b0a9-920332da4dc6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:03.396497 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.396472 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-podinfo" pod "e2b7b8df-514a-43c7-b0a9-920332da4dc6" (UID: "e2b7b8df-514a-43c7-b0a9-920332da4dc6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:46:03.396706 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.396682 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-envoy" pod "e2b7b8df-514a-43c7-b0a9-920332da4dc6" (UID: "e2b7b8df-514a-43c7-b0a9-920332da4dc6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:03.425761 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.425728 5904 scope.go:122] "RemoveContainer" containerID="58c14dd957c5241fdd00d493cfcf99606fce04f839e6115972d633ab3f2dfd82" Nov 04 05:46:03.426119 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:46:03.426100 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58c14dd957c5241fdd00d493cfcf99606fce04f839e6115972d633ab3f2dfd82\": container with ID starting with 58c14dd957c5241fdd00d493cfcf99606fce04f839e6115972d633ab3f2dfd82 not found: ID does not exist" containerID="58c14dd957c5241fdd00d493cfcf99606fce04f839e6115972d633ab3f2dfd82" Nov 04 05:46:03.426218 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.426128 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58c14dd957c5241fdd00d493cfcf99606fce04f839e6115972d633ab3f2dfd82"} err="failed to get container status \"58c14dd957c5241fdd00d493cfcf99606fce04f839e6115972d633ab3f2dfd82\": rpc error: code = NotFound desc = could not find container \"58c14dd957c5241fdd00d493cfcf99606fce04f839e6115972d633ab3f2dfd82\": container with ID starting with 58c14dd957c5241fdd00d493cfcf99606fce04f839e6115972d633ab3f2dfd82 not found: ID does not exist" Nov 04 05:46:03.485403 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.485375 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:46:03.495151 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.495113 5904 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-envoy\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.495151 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.495140 5904 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-credential-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.495151 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.495149 5904 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istiod-ca-cert\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.495151 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.495159 5904 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-workload-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.495472 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.495169 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tlrn\" (UniqueName: \"kubernetes.io/projected/e2b7b8df-514a-43c7-b0a9-920332da4dc6-kube-api-access-9tlrn\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.495472 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.495178 5904 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-data\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.495472 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.495188 5904 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b7b8df-514a-43c7-b0a9-920332da4dc6-workload-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.495472 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.495196 5904 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-podinfo\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.495472 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.495205 5904 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2b7b8df-514a-43c7-b0a9-920332da4dc6-istio-token\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.587183 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.587157 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:46:03.595973 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.595944 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-podinfo\") pod \"353f0a70-20f0-4878-b7fa-20221f62ebc4\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " Nov 04 05:46:03.596099 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.596004 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-envoy\") pod \"353f0a70-20f0-4878-b7fa-20221f62ebc4\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " Nov 04 05:46:03.596099 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.596035 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-credential-socket\") pod \"353f0a70-20f0-4878-b7fa-20221f62ebc4\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " Nov 04 05:46:03.596099 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.596080 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-data\") pod \"353f0a70-20f0-4878-b7fa-20221f62ebc4\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " Nov 04 05:46:03.596239 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.596152 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/353f0a70-20f0-4878-b7fa-20221f62ebc4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/353f0a70-20f0-4878-b7fa-20221f62ebc4-istiod-ca-cert\") pod \"353f0a70-20f0-4878-b7fa-20221f62ebc4\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " Nov 04 05:46:03.596239 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.596180 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/353f0a70-20f0-4878-b7fa-20221f62ebc4-kube-api-access-dqfvd\" (UniqueName: \"kubernetes.io/projected/353f0a70-20f0-4878-b7fa-20221f62ebc4-kube-api-access-dqfvd\") pod \"353f0a70-20f0-4878-b7fa-20221f62ebc4\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " Nov 04 05:46:03.596356 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.596216 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-workload-certs\") pod \"353f0a70-20f0-4878-b7fa-20221f62ebc4\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " Nov 04 05:46:03.596356 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.596326 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-token\" (UniqueName: \"kubernetes.io/projected/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-token\") pod \"353f0a70-20f0-4878-b7fa-20221f62ebc4\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " Nov 04 05:46:03.596356 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.596345 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-workload-socket\") pod \"353f0a70-20f0-4878-b7fa-20221f62ebc4\" (UID: \"353f0a70-20f0-4878-b7fa-20221f62ebc4\") " Nov 04 05:46:03.596517 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.596400 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-credential-socket" pod "353f0a70-20f0-4878-b7fa-20221f62ebc4" (UID: "353f0a70-20f0-4878-b7fa-20221f62ebc4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:03.596720 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.596684 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/353f0a70-20f0-4878-b7fa-20221f62ebc4-istiod-ca-cert" pod "353f0a70-20f0-4878-b7fa-20221f62ebc4" (UID: "353f0a70-20f0-4878-b7fa-20221f62ebc4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:03.596831 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.596715 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-workload-certs" pod "353f0a70-20f0-4878-b7fa-20221f62ebc4" (UID: "353f0a70-20f0-4878-b7fa-20221f62ebc4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:03.596831 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.596793 5904 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-credential-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.596831 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.596813 5904 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/353f0a70-20f0-4878-b7fa-20221f62ebc4-istiod-ca-cert\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.597005 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.596934 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-workload-socket" pod "353f0a70-20f0-4878-b7fa-20221f62ebc4" (UID: "353f0a70-20f0-4878-b7fa-20221f62ebc4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:03.597358 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.597332 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-data" pod "353f0a70-20f0-4878-b7fa-20221f62ebc4" (UID: "353f0a70-20f0-4878-b7fa-20221f62ebc4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:03.598488 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.598469 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/353f0a70-20f0-4878-b7fa-20221f62ebc4-kube-api-access-dqfvd" pod "353f0a70-20f0-4878-b7fa-20221f62ebc4" (UID: "353f0a70-20f0-4878-b7fa-20221f62ebc4"). InnerVolumeSpecName "kube-api-access-dqfvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:03.598605 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.598582 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-token" pod "353f0a70-20f0-4878-b7fa-20221f62ebc4" (UID: "353f0a70-20f0-4878-b7fa-20221f62ebc4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:03.598683 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.598651 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-podinfo" pod "353f0a70-20f0-4878-b7fa-20221f62ebc4" (UID: "353f0a70-20f0-4878-b7fa-20221f62ebc4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:46:03.598683 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.598664 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-envoy" pod "353f0a70-20f0-4878-b7fa-20221f62ebc4" (UID: "353f0a70-20f0-4878-b7fa-20221f62ebc4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:03.675927 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.675890 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw"] Nov 04 05:46:03.677780 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.677757 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-h9rbw"] Nov 04 05:46:03.697539 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.697508 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-workload-certs\") pod \"792505fe-6c68-4def-bd32-a41ca2a1fb34\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " Nov 04 05:46:03.697671 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.697563 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-data\") pod \"792505fe-6c68-4def-bd32-a41ca2a1fb34\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " Nov 04 05:46:03.697671 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.697655 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/792505fe-6c68-4def-bd32-a41ca2a1fb34-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/792505fe-6c68-4def-bd32-a41ca2a1fb34-istiod-ca-cert\") pod \"792505fe-6c68-4def-bd32-a41ca2a1fb34\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " Nov 04 05:46:03.697802 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.697693 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-token\" (UniqueName: \"kubernetes.io/projected/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-token\") pod \"792505fe-6c68-4def-bd32-a41ca2a1fb34\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " Nov 04 05:46:03.697802 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.697726 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-workload-socket\") pod \"792505fe-6c68-4def-bd32-a41ca2a1fb34\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " Nov 04 05:46:03.697802 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.697759 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-envoy\") pod \"792505fe-6c68-4def-bd32-a41ca2a1fb34\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " Nov 04 05:46:03.697802 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.697798 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-podinfo\") pod \"792505fe-6c68-4def-bd32-a41ca2a1fb34\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " Nov 04 05:46:03.697997 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.697825 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/792505fe-6c68-4def-bd32-a41ca2a1fb34-kube-api-access-swfkg\" (UniqueName: \"kubernetes.io/projected/792505fe-6c68-4def-bd32-a41ca2a1fb34-kube-api-access-swfkg\") pod \"792505fe-6c68-4def-bd32-a41ca2a1fb34\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " Nov 04 05:46:03.697997 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.697844 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-workload-certs" pod "792505fe-6c68-4def-bd32-a41ca2a1fb34" (UID: "792505fe-6c68-4def-bd32-a41ca2a1fb34"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:03.697997 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.697855 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-data" pod "792505fe-6c68-4def-bd32-a41ca2a1fb34" (UID: "792505fe-6c68-4def-bd32-a41ca2a1fb34"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:03.697997 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.697856 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-credential-socket\") pod \"792505fe-6c68-4def-bd32-a41ca2a1fb34\" (UID: \"792505fe-6c68-4def-bd32-a41ca2a1fb34\") " Nov 04 05:46:03.698202 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.698003 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-workload-socket" pod "792505fe-6c68-4def-bd32-a41ca2a1fb34" (UID: "792505fe-6c68-4def-bd32-a41ca2a1fb34"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:03.698202 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.698087 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/792505fe-6c68-4def-bd32-a41ca2a1fb34-istiod-ca-cert" pod "792505fe-6c68-4def-bd32-a41ca2a1fb34" (UID: "792505fe-6c68-4def-bd32-a41ca2a1fb34"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:03.698202 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.698105 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-credential-socket" pod "792505fe-6c68-4def-bd32-a41ca2a1fb34" (UID: "792505fe-6c68-4def-bd32-a41ca2a1fb34"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:03.698432 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.698407 5904 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-data\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.698555 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.698434 5904 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/792505fe-6c68-4def-bd32-a41ca2a1fb34-istiod-ca-cert\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.698555 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.698452 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqfvd\" (UniqueName: \"kubernetes.io/projected/353f0a70-20f0-4878-b7fa-20221f62ebc4-kube-api-access-dqfvd\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.698555 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.698465 5904 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-workload-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.698555 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.698478 5904 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-workload-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.698555 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.698486 5904 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-token\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.698555 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.698495 5904 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-workload-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.698555 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.698503 5904 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-credential-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.698555 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.698512 5904 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-podinfo\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.698555 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.698524 5904 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-workload-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.698555 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.698535 5904 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/353f0a70-20f0-4878-b7fa-20221f62ebc4-istio-envoy\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.698555 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.698544 5904 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-data\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.699906 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.699884 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-token" pod "792505fe-6c68-4def-bd32-a41ca2a1fb34" (UID: "792505fe-6c68-4def-bd32-a41ca2a1fb34"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:03.700157 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.700135 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-podinfo" pod "792505fe-6c68-4def-bd32-a41ca2a1fb34" (UID: "792505fe-6c68-4def-bd32-a41ca2a1fb34"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:46:03.700434 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.700415 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/792505fe-6c68-4def-bd32-a41ca2a1fb34-kube-api-access-swfkg" pod "792505fe-6c68-4def-bd32-a41ca2a1fb34" (UID: "792505fe-6c68-4def-bd32-a41ca2a1fb34"). InnerVolumeSpecName "kube-api-access-swfkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:03.700434 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.700427 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-envoy" pod "792505fe-6c68-4def-bd32-a41ca2a1fb34" (UID: "792505fe-6c68-4def-bd32-a41ca2a1fb34"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:03.799987 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.799948 5904 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-envoy\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.799987 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.799976 5904 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-podinfo\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.799987 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.799986 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swfkg\" (UniqueName: \"kubernetes.io/projected/792505fe-6c68-4def-bd32-a41ca2a1fb34-kube-api-access-swfkg\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:03.799987 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:03.799996 5904 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/792505fe-6c68-4def-bd32-a41ca2a1fb34-istio-token\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:04.365722 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.365692 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" Nov 04 05:46:04.366151 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.365727 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw" event={"ID":"353f0a70-20f0-4878-b7fa-20221f62ebc4","Type":"ContainerDied","Data":"85cc0fc908925fcf4c807e03f5742ba84f0e0ef6acd05dccac3e986bde5eb056"} Nov 04 05:46:04.366151 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.365796 5904 scope.go:122] "RemoveContainer" containerID="f3a472abe60d6635427e9e78bce3afe26e220060e57ed1034439baeeb84b8406" Nov 04 05:46:04.367218 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.367178 5904 generic.go:356] "Generic (PLEG): container finished" podID="792505fe-6c68-4def-bd32-a41ca2a1fb34" containerID="a37b8db06f38ce129bd02e7746546b61a65a249afca3ea1ccd1eb9f1b62e293f" exitCode=0 Nov 04 05:46:04.367218 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.367213 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" event={"ID":"792505fe-6c68-4def-bd32-a41ca2a1fb34","Type":"ContainerDied","Data":"a37b8db06f38ce129bd02e7746546b61a65a249afca3ea1ccd1eb9f1b62e293f"} Nov 04 05:46:04.367348 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.367248 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" Nov 04 05:46:04.367348 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.367277 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4" event={"ID":"792505fe-6c68-4def-bd32-a41ca2a1fb34","Type":"ContainerDied","Data":"3108a41296314ad3bf491872153df5788dbfa4a06bfdd55236adf9f29af4a575"} Nov 04 05:46:04.377578 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.377564 5904 scope.go:122] "RemoveContainer" containerID="a37b8db06f38ce129bd02e7746546b61a65a249afca3ea1ccd1eb9f1b62e293f" Nov 04 05:46:04.389078 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.389030 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw"] Nov 04 05:46:04.394401 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.394380 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-m5shw"] Nov 04 05:46:04.429032 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.429008 5904 scope.go:122] "RemoveContainer" containerID="a37b8db06f38ce129bd02e7746546b61a65a249afca3ea1ccd1eb9f1b62e293f" Nov 04 05:46:04.429380 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:46:04.429355 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a37b8db06f38ce129bd02e7746546b61a65a249afca3ea1ccd1eb9f1b62e293f\": container with ID starting with a37b8db06f38ce129bd02e7746546b61a65a249afca3ea1ccd1eb9f1b62e293f not found: ID does not exist" containerID="a37b8db06f38ce129bd02e7746546b61a65a249afca3ea1ccd1eb9f1b62e293f" Nov 04 05:46:04.429466 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.429387 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a37b8db06f38ce129bd02e7746546b61a65a249afca3ea1ccd1eb9f1b62e293f"} err="failed to get container status \"a37b8db06f38ce129bd02e7746546b61a65a249afca3ea1ccd1eb9f1b62e293f\": rpc error: code = NotFound desc = could not find container \"a37b8db06f38ce129bd02e7746546b61a65a249afca3ea1ccd1eb9f1b62e293f\": container with ID starting with a37b8db06f38ce129bd02e7746546b61a65a249afca3ea1ccd1eb9f1b62e293f not found: ID does not exist" Nov 04 05:46:04.439896 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.439868 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4"] Nov 04 05:46:04.443312 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.443288 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-27935d-openshift-default-6cb57f858f-ffrs4"] Nov 04 05:46:04.543718 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.543690 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:46:04.608011 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.607978 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b00d56d-9587-42b9-81a1-1d3cb9787842-kube-api-access-bzk62\" (UniqueName: \"kubernetes.io/projected/0b00d56d-9587-42b9-81a1-1d3cb9787842-kube-api-access-bzk62\") pod \"0b00d56d-9587-42b9-81a1-1d3cb9787842\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " Nov 04 05:46:04.608170 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.608032 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-workload-socket\") pod \"0b00d56d-9587-42b9-81a1-1d3cb9787842\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " Nov 04 05:46:04.608170 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.608064 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-envoy\") pod \"0b00d56d-9587-42b9-81a1-1d3cb9787842\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " Nov 04 05:46:04.608327 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.608255 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-workload-certs\") pod \"0b00d56d-9587-42b9-81a1-1d3cb9787842\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " Nov 04 05:46:04.608423 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.608371 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-data\") pod \"0b00d56d-9587-42b9-81a1-1d3cb9787842\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " Nov 04 05:46:04.608423 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.608393 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-workload-socket" pod "0b00d56d-9587-42b9-81a1-1d3cb9787842" (UID: "0b00d56d-9587-42b9-81a1-1d3cb9787842"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:04.608549 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.608423 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-credential-socket\") pod \"0b00d56d-9587-42b9-81a1-1d3cb9787842\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " Nov 04 05:46:04.608549 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.608439 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-workload-certs" pod "0b00d56d-9587-42b9-81a1-1d3cb9787842" (UID: "0b00d56d-9587-42b9-81a1-1d3cb9787842"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:04.608549 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.608486 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-podinfo\") pod \"0b00d56d-9587-42b9-81a1-1d3cb9787842\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " Nov 04 05:46:04.608549 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.608506 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-token\" (UniqueName: \"kubernetes.io/projected/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-token\") pod \"0b00d56d-9587-42b9-81a1-1d3cb9787842\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " Nov 04 05:46:04.608750 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.608572 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b00d56d-9587-42b9-81a1-1d3cb9787842-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b00d56d-9587-42b9-81a1-1d3cb9787842-istiod-ca-cert\") pod \"0b00d56d-9587-42b9-81a1-1d3cb9787842\" (UID: \"0b00d56d-9587-42b9-81a1-1d3cb9787842\") " Nov 04 05:46:04.608750 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.608610 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-data" pod "0b00d56d-9587-42b9-81a1-1d3cb9787842" (UID: "0b00d56d-9587-42b9-81a1-1d3cb9787842"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:04.608750 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.608680 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-credential-socket" pod "0b00d56d-9587-42b9-81a1-1d3cb9787842" (UID: "0b00d56d-9587-42b9-81a1-1d3cb9787842"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:04.609017 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.608998 5904 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-workload-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:04.609090 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.609023 5904 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-workload-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:04.609090 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.609032 5904 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-data\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:04.609090 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.609041 5904 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-credential-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:04.609090 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.608996 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b00d56d-9587-42b9-81a1-1d3cb9787842-istiod-ca-cert" pod "0b00d56d-9587-42b9-81a1-1d3cb9787842" (UID: "0b00d56d-9587-42b9-81a1-1d3cb9787842"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:04.610346 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.610319 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-envoy" pod "0b00d56d-9587-42b9-81a1-1d3cb9787842" (UID: "0b00d56d-9587-42b9-81a1-1d3cb9787842"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:04.610443 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.610422 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b00d56d-9587-42b9-81a1-1d3cb9787842-kube-api-access-bzk62" pod "0b00d56d-9587-42b9-81a1-1d3cb9787842" (UID: "0b00d56d-9587-42b9-81a1-1d3cb9787842"). InnerVolumeSpecName "kube-api-access-bzk62". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:04.610614 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.610594 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-podinfo" pod "0b00d56d-9587-42b9-81a1-1d3cb9787842" (UID: "0b00d56d-9587-42b9-81a1-1d3cb9787842"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:46:04.610806 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.610784 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-token" pod "0b00d56d-9587-42b9-81a1-1d3cb9787842" (UID: "0b00d56d-9587-42b9-81a1-1d3cb9787842"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:04.710209 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.710110 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzk62\" (UniqueName: \"kubernetes.io/projected/0b00d56d-9587-42b9-81a1-1d3cb9787842-kube-api-access-bzk62\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:04.710209 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.710139 5904 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-envoy\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:04.710209 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.710153 5904 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-podinfo\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:04.710209 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.710164 5904 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b00d56d-9587-42b9-81a1-1d3cb9787842-istio-token\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:04.710209 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:04.710176 5904 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b00d56d-9587-42b9-81a1-1d3cb9787842-istiod-ca-cert\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.170679 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.170646 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="353f0a70-20f0-4878-b7fa-20221f62ebc4" path="/var/lib/kubelet/pods/353f0a70-20f0-4878-b7fa-20221f62ebc4/volumes" Nov 04 05:46:05.172256 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.171297 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="792505fe-6c68-4def-bd32-a41ca2a1fb34" path="/var/lib/kubelet/pods/792505fe-6c68-4def-bd32-a41ca2a1fb34/volumes" Nov 04 05:46:05.172256 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.171876 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2b7b8df-514a-43c7-b0a9-920332da4dc6" path="/var/lib/kubelet/pods/e2b7b8df-514a-43c7-b0a9-920332da4dc6/volumes" Nov 04 05:46:05.340040 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.339959 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:46:05.378758 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.378727 5904 generic.go:356] "Generic (PLEG): container finished" podID="9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" containerID="14dbca9f8afa7e052892723c18f4180fb109345487d76750076c6ccc711ae626" exitCode=0 Nov 04 05:46:05.379142 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.378796 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" event={"ID":"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42","Type":"ContainerDied","Data":"14dbca9f8afa7e052892723c18f4180fb109345487d76750076c6ccc711ae626"} Nov 04 05:46:05.380519 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.380492 5904 generic.go:356] "Generic (PLEG): container finished" podID="0b00d56d-9587-42b9-81a1-1d3cb9787842" containerID="8fff9c9f288cbcbd0000c74ddd2bc4f8b9a1ca332d2390ecd4e97ecc60838467" exitCode=0 Nov 04 05:46:05.380655 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.380579 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" event={"ID":"0b00d56d-9587-42b9-81a1-1d3cb9787842","Type":"ContainerDied","Data":"8fff9c9f288cbcbd0000c74ddd2bc4f8b9a1ca332d2390ecd4e97ecc60838467"} Nov 04 05:46:05.380655 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.380589 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" Nov 04 05:46:05.380763 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.380658 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p" event={"ID":"0b00d56d-9587-42b9-81a1-1d3cb9787842","Type":"ContainerDied","Data":"252ad03a61f4ed9f14818499b536e3ea645c3dfb6e990d9ed4ec59d1cebc1e7f"} Nov 04 05:46:05.380763 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.380683 5904 scope.go:122] "RemoveContainer" containerID="8fff9c9f288cbcbd0000c74ddd2bc4f8b9a1ca332d2390ecd4e97ecc60838467" Nov 04 05:46:05.382283 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.382257 5904 generic.go:356] "Generic (PLEG): container finished" podID="c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" containerID="1dbb4f310e1a36a5344e20bb1e0536f1b611e42022b1d5095d84e32e4274989b" exitCode=0 Nov 04 05:46:05.382392 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.382356 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" Nov 04 05:46:05.382469 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.382458 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" event={"ID":"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1","Type":"ContainerDied","Data":"1dbb4f310e1a36a5344e20bb1e0536f1b611e42022b1d5095d84e32e4274989b"} Nov 04 05:46:05.382534 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.382484 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg" event={"ID":"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1","Type":"ContainerDied","Data":"fc51d7e589c2eec434313aff359a08c6bbc1a1f32fc63ada12500f3162eef20f"} Nov 04 05:46:05.383488 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.383466 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:46:05.384824 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.384804 5904 generic.go:356] "Generic (PLEG): container finished" podID="1f4fab8f-362d-4834-bd81-a139841c6bcb" containerID="3cc6637e2eaf10b27b6df0bdf113ecbd01b947c368f788bb49637a410755e3a9" exitCode=0 Nov 04 05:46:05.384904 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.384883 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" event={"ID":"1f4fab8f-362d-4834-bd81-a139841c6bcb","Type":"ContainerDied","Data":"3cc6637e2eaf10b27b6df0bdf113ecbd01b947c368f788bb49637a410755e3a9"} Nov 04 05:46:05.390761 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.390740 5904 scope.go:122] "RemoveContainer" containerID="8fff9c9f288cbcbd0000c74ddd2bc4f8b9a1ca332d2390ecd4e97ecc60838467" Nov 04 05:46:05.391072 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:46:05.391051 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fff9c9f288cbcbd0000c74ddd2bc4f8b9a1ca332d2390ecd4e97ecc60838467\": container with ID starting with 8fff9c9f288cbcbd0000c74ddd2bc4f8b9a1ca332d2390ecd4e97ecc60838467 not found: ID does not exist" containerID="8fff9c9f288cbcbd0000c74ddd2bc4f8b9a1ca332d2390ecd4e97ecc60838467" Nov 04 05:46:05.391157 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.391077 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fff9c9f288cbcbd0000c74ddd2bc4f8b9a1ca332d2390ecd4e97ecc60838467"} err="failed to get container status \"8fff9c9f288cbcbd0000c74ddd2bc4f8b9a1ca332d2390ecd4e97ecc60838467\": rpc error: code = NotFound desc = could not find container \"8fff9c9f288cbcbd0000c74ddd2bc4f8b9a1ca332d2390ecd4e97ecc60838467\": container with ID starting with 8fff9c9f288cbcbd0000c74ddd2bc4f8b9a1ca332d2390ecd4e97ecc60838467 not found: ID does not exist" Nov 04 05:46:05.391157 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.391099 5904 scope.go:122] "RemoveContainer" containerID="1dbb4f310e1a36a5344e20bb1e0536f1b611e42022b1d5095d84e32e4274989b" Nov 04 05:46:05.400950 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.400928 5904 scope.go:122] "RemoveContainer" containerID="1dbb4f310e1a36a5344e20bb1e0536f1b611e42022b1d5095d84e32e4274989b" Nov 04 05:46:05.401268 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:46:05.401215 5904 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dbb4f310e1a36a5344e20bb1e0536f1b611e42022b1d5095d84e32e4274989b\": container with ID starting with 1dbb4f310e1a36a5344e20bb1e0536f1b611e42022b1d5095d84e32e4274989b not found: ID does not exist" containerID="1dbb4f310e1a36a5344e20bb1e0536f1b611e42022b1d5095d84e32e4274989b" Nov 04 05:46:05.401370 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.401277 5904 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dbb4f310e1a36a5344e20bb1e0536f1b611e42022b1d5095d84e32e4274989b"} err="failed to get container status \"1dbb4f310e1a36a5344e20bb1e0536f1b611e42022b1d5095d84e32e4274989b\": rpc error: code = NotFound desc = could not find container \"1dbb4f310e1a36a5344e20bb1e0536f1b611e42022b1d5095d84e32e4274989b\": container with ID starting with 1dbb4f310e1a36a5344e20bb1e0536f1b611e42022b1d5095d84e32e4274989b not found: ID does not exist" Nov 04 05:46:05.401370 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.401301 5904 scope.go:122] "RemoveContainer" containerID="3cc6637e2eaf10b27b6df0bdf113ecbd01b947c368f788bb49637a410755e3a9" Nov 04 05:46:05.418083 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.417712 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istiod-ca-cert\") pod \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " Nov 04 05:46:05.418083 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.417753 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-podinfo\") pod \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " Nov 04 05:46:05.418083 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.417822 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-token\" (UniqueName: \"kubernetes.io/projected/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-token\") pod \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " Nov 04 05:46:05.418083 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.417894 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-kube-api-access-qsh2n\" (UniqueName: \"kubernetes.io/projected/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-kube-api-access-qsh2n\") pod \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " Nov 04 05:46:05.418083 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.417958 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-workload-certs\") pod \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " Nov 04 05:46:05.418083 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.417992 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-data\") pod \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " Nov 04 05:46:05.418083 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.418030 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-envoy\") pod \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " Nov 04 05:46:05.418083 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.418080 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-credential-socket\") pod \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " Nov 04 05:46:05.419086 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.418146 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-workload-socket\") pod \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\" (UID: \"c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1\") " Nov 04 05:46:05.419086 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.418771 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-workload-socket" pod "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" (UID: "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:05.419197 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.419123 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istiod-ca-cert" pod "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" (UID: "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:05.423326 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.419916 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-credential-socket" pod "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" (UID: "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:05.423326 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.420108 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-data" pod "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" (UID: "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:05.423326 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.420431 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-workload-certs" pod "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" (UID: "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:05.423326 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.422333 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-podinfo" pod "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" (UID: "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:46:05.423326 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.423000 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-token" pod "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" (UID: "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:05.423326 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.423172 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p"] Nov 04 05:46:05.425551 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.425531 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-gdm9p"] Nov 04 05:46:05.425941 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.425921 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-kube-api-access-qsh2n" pod "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" (UID: "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1"). InnerVolumeSpecName "kube-api-access-qsh2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:05.427077 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.427058 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-envoy" pod "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" (UID: "c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:05.504358 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.504334 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:46:05.519831 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.519810 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f4fab8f-362d-4834-bd81-a139841c6bcb-kube-api-access-nwbln\" (UniqueName: \"kubernetes.io/projected/1f4fab8f-362d-4834-bd81-a139841c6bcb-kube-api-access-nwbln\") pod \"1f4fab8f-362d-4834-bd81-a139841c6bcb\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " Nov 04 05:46:05.519955 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.519841 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-workload-certs\") pod \"1f4fab8f-362d-4834-bd81-a139841c6bcb\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " Nov 04 05:46:05.519955 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.519935 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-data\") pod \"1f4fab8f-362d-4834-bd81-a139841c6bcb\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " Nov 04 05:46:05.520068 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.519958 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-podinfo\") pod \"1f4fab8f-362d-4834-bd81-a139841c6bcb\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " Nov 04 05:46:05.520068 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.519981 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f4fab8f-362d-4834-bd81-a139841c6bcb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f4fab8f-362d-4834-bd81-a139841c6bcb-istiod-ca-cert\") pod \"1f4fab8f-362d-4834-bd81-a139841c6bcb\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " Nov 04 05:46:05.520068 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520010 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-credential-socket\") pod \"1f4fab8f-362d-4834-bd81-a139841c6bcb\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " Nov 04 05:46:05.520068 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520048 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-token\" (UniqueName: \"kubernetes.io/projected/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-token\") pod \"1f4fab8f-362d-4834-bd81-a139841c6bcb\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " Nov 04 05:46:05.520298 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520077 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-envoy\") pod \"1f4fab8f-362d-4834-bd81-a139841c6bcb\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " Nov 04 05:46:05.520298 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520141 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-workload-socket\") pod \"1f4fab8f-362d-4834-bd81-a139841c6bcb\" (UID: \"1f4fab8f-362d-4834-bd81-a139841c6bcb\") " Nov 04 05:46:05.520298 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520187 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-data" pod "1f4fab8f-362d-4834-bd81-a139841c6bcb" (UID: "1f4fab8f-362d-4834-bd81-a139841c6bcb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:05.520298 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520207 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-workload-certs" pod "1f4fab8f-362d-4834-bd81-a139841c6bcb" (UID: "1f4fab8f-362d-4834-bd81-a139841c6bcb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:05.520505 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520322 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-credential-socket" pod "1f4fab8f-362d-4834-bd81-a139841c6bcb" (UID: "1f4fab8f-362d-4834-bd81-a139841c6bcb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:05.520505 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520458 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-workload-socket" pod "1f4fab8f-362d-4834-bd81-a139841c6bcb" (UID: "1f4fab8f-362d-4834-bd81-a139841c6bcb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:05.520617 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520600 5904 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-credential-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.520671 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520623 5904 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-credential-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.520671 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520637 5904 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-workload-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.520671 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520650 5904 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istiod-ca-cert\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.520671 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520663 5904 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-workload-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.520864 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520676 5904 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-podinfo\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.520864 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520691 5904 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-workload-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.520864 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520704 5904 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-token\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.520864 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520718 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsh2n\" (UniqueName: \"kubernetes.io/projected/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-kube-api-access-qsh2n\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.520864 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520673 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f4fab8f-362d-4834-bd81-a139841c6bcb-istiod-ca-cert" pod "1f4fab8f-362d-4834-bd81-a139841c6bcb" (UID: "1f4fab8f-362d-4834-bd81-a139841c6bcb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:05.520864 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520767 5904 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-workload-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.520864 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520784 5904 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-data\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.520864 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520797 5904 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-data\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.520864 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.520810 5904 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1-istio-envoy\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.523697 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.522815 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-podinfo" pod "1f4fab8f-362d-4834-bd81-a139841c6bcb" (UID: "1f4fab8f-362d-4834-bd81-a139841c6bcb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:46:05.523697 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.523573 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-token" pod "1f4fab8f-362d-4834-bd81-a139841c6bcb" (UID: "1f4fab8f-362d-4834-bd81-a139841c6bcb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:05.524186 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.524153 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-envoy" pod "1f4fab8f-362d-4834-bd81-a139841c6bcb" (UID: "1f4fab8f-362d-4834-bd81-a139841c6bcb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:05.524652 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.524628 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f4fab8f-362d-4834-bd81-a139841c6bcb-kube-api-access-nwbln" pod "1f4fab8f-362d-4834-bd81-a139841c6bcb" (UID: "1f4fab8f-362d-4834-bd81-a139841c6bcb"). InnerVolumeSpecName "kube-api-access-nwbln". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:05.622047 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.621976 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-workload-socket\") pod \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " Nov 04 05:46:05.622047 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622030 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-envoy\") pod \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " Nov 04 05:46:05.622208 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622052 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istiod-ca-cert\") pod \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " Nov 04 05:46:05.622208 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622088 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-token\" (UniqueName: \"kubernetes.io/projected/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-token\") pod \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " Nov 04 05:46:05.622208 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622110 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-workload-certs\") pod \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " Nov 04 05:46:05.622208 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622169 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-kube-api-access-lqptk\" (UniqueName: \"kubernetes.io/projected/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-kube-api-access-lqptk\") pod \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " Nov 04 05:46:05.622208 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622188 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-data\") pod \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " Nov 04 05:46:05.622911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622221 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-podinfo\") pod \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " Nov 04 05:46:05.622911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622279 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-workload-socket" pod "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" (UID: "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:05.622911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622314 5904 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-credential-socket\") pod \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\" (UID: \"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42\") " Nov 04 05:46:05.622911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622521 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-data" pod "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" (UID: "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:05.622911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622541 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-workload-certs" pod "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" (UID: "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:05.622911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622560 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istiod-ca-cert" pod "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" (UID: "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:05.622911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622692 5904 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-workload-certs\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.622911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622703 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-credential-socket" pod "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" (UID: "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:05.622911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622712 5904 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-podinfo\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.622911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622727 5904 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f4fab8f-362d-4834-bd81-a139841c6bcb-istiod-ca-cert\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.622911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622738 5904 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-data\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.622911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622747 5904 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-token\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.622911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622755 5904 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f4fab8f-362d-4834-bd81-a139841c6bcb-istio-envoy\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.622911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622764 5904 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-workload-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.622911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622773 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwbln\" (UniqueName: \"kubernetes.io/projected/1f4fab8f-362d-4834-bd81-a139841c6bcb-kube-api-access-nwbln\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.622911 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.622781 5904 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istiod-ca-cert\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.624171 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.624152 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-token" pod "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" (UID: "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:05.624483 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.624464 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-podinfo" pod "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" (UID: "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:46:05.624654 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.624637 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-envoy" pod "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" (UID: "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:05.624718 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.624664 5904 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-kube-api-access-lqptk" pod "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" (UID: "9d600ba3-f1dd-4cf8-a2d2-208bc502aa42"). InnerVolumeSpecName "kube-api-access-lqptk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:05.704003 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.703967 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg"] Nov 04 05:46:05.706948 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.706922 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-nm9fg"] Nov 04 05:46:05.723753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.723732 5904 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqptk\" (UniqueName: \"kubernetes.io/projected/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-kube-api-access-lqptk\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.723753 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.723754 5904 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-podinfo\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.723877 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.723764 5904 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-credential-socket\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.723877 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.723773 5904 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-envoy\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:05.723877 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:05.723782 5904 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42-istio-token\") on node \"ip-10-0-128-126.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:06.391739 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:06.391658 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" Nov 04 05:46:06.391739 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:06.391694 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk" event={"ID":"9d600ba3-f1dd-4cf8-a2d2-208bc502aa42","Type":"ContainerDied","Data":"87b672981a3bd5a1ab82466a2c377e157a957d4caeebf426f5be915537663007"} Nov 04 05:46:06.392280 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:06.391758 5904 scope.go:122] "RemoveContainer" containerID="14dbca9f8afa7e052892723c18f4180fb109345487d76750076c6ccc711ae626" Nov 04 05:46:06.394436 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:06.394411 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" event={"ID":"1f4fab8f-362d-4834-bd81-a139841c6bcb","Type":"ContainerDied","Data":"3b896263928aaa02d83a695909e0873b4f144d449896fc52b8b2f788cfb0dbca"} Nov 04 05:46:06.394554 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:06.394481 5904 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg" Nov 04 05:46:06.415912 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:06.415889 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk"] Nov 04 05:46:06.419083 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:06.419064 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-9d808e-openshift-default-54986c8d59-995vk"] Nov 04 05:46:06.429150 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:06.429125 5904 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg"] Nov 04 05:46:06.432422 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:06.432404 5904 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-f910f5-openshift-default-7fc8dc77d5-sknfg"] Nov 04 05:46:07.167272 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:07.167221 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b00d56d-9587-42b9-81a1-1d3cb9787842" path="/var/lib/kubelet/pods/0b00d56d-9587-42b9-81a1-1d3cb9787842/volumes" Nov 04 05:46:07.167678 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:07.167666 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f4fab8f-362d-4834-bd81-a139841c6bcb" path="/var/lib/kubelet/pods/1f4fab8f-362d-4834-bd81-a139841c6bcb/volumes" Nov 04 05:46:07.168047 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:07.168035 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" path="/var/lib/kubelet/pods/9d600ba3-f1dd-4cf8-a2d2-208bc502aa42/volumes" Nov 04 05:46:07.168737 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:07.168723 5904 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" path="/var/lib/kubelet/pods/c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1/volumes" Nov 04 05:46:12.302765 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:12.302736 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-9m4gc_95e8955f-5e49-4bfb-8511-d2a57effca60/manager/0.log" Nov 04 05:46:12.467248 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:12.467202 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-nrwnc_4c46526a-a778-4be1-bdf7-fc3c1f7d38b2/manager/0.log" Nov 04 05:46:12.657611 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:12.657585 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-l8cn6_1f43099e-9c80-4bc5-9a7a-18d3e22534f9/manager/0.log" Nov 04 05:46:12.875208 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:12.875180 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-595689b685-gmspv_57c93381-b98d-4d00-baf7-1b31d0f6c433/manager/1.log" Nov 04 05:46:13.488063 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:13.488037 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-mkgsm_36073709-5d75-4ba9-9adf-fb35c540843b/proxy/0.log" Nov 04 05:46:14.076241 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:14.076195 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-595689b685-gmspv_57c93381-b98d-4d00-baf7-1b31d0f6c433/manager/0.log" Nov 04 05:46:14.625198 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:14.625157 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5d8dd6bcb9-tbdvp_c8ebad6f-24c6-4160-b09b-4be3c3bec639/maas-api/0.log" Nov 04 05:46:17.302948 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:17.302920 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-2bpbj_2b8a322c-d8e3-4457-bce3-bf609408418e/postgres/0.log" Nov 04 05:46:19.691064 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:19.691036 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-wm6vh_f31c8bc8-3733-4b18-82fb-a7273a6300df/kuadrant-console-plugin/0.log" Nov 04 05:46:22.578378 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:22.578352 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5488c58ff5-nc2z6_b5548164-9095-44ae-820c-dbddc724ecbf/kube-auth-proxy/0.log" Nov 04 05:46:23.645125 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:23.645094 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-7rc4s_87f3858d-ee78-4468-8b48-16073a80b645/istio-proxy/0.log" Nov 04 05:46:23.996120 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:23.996042 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-hwbs4_8ca67e76-2ec7-4142-a8c3-0493d3a1cc63/istio-proxy/0.log" Nov 04 05:46:24.354807 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:24.354776 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-zfft9_c671ccc5-92a6-4b7d-8a92-7955bd3e15fd/istio-proxy/0.log" Nov 04 05:46:27.132194 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:46:27.132150 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:46:37.688376 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:37.688341 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-9m4gc_95e8955f-5e49-4bfb-8511-d2a57effca60/manager/0.log" Nov 04 05:46:37.867797 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:37.867766 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-nrwnc_4c46526a-a778-4be1-bdf7-fc3c1f7d38b2/manager/0.log" Nov 04 05:46:38.058279 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:38.058188 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-l8cn6_1f43099e-9c80-4bc5-9a7a-18d3e22534f9/manager/0.log" Nov 04 05:46:38.269398 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:38.269367 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-595689b685-gmspv_57c93381-b98d-4d00-baf7-1b31d0f6c433/manager/1.log" Nov 04 05:46:38.881845 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:38.881812 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-mkgsm_36073709-5d75-4ba9-9adf-fb35c540843b/proxy/0.log" Nov 04 05:46:39.381573 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:39.381539 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5d8dd6bcb9-tbdvp_c8ebad6f-24c6-4160-b09b-4be3c3bec639/maas-api/0.log" Nov 04 05:46:41.951272 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:41.951244 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-2bpbj_2b8a322c-d8e3-4457-bce3-bf609408418e/postgres/0.log" Nov 04 05:46:43.617377 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:43.617349 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-wm6vh_f31c8bc8-3733-4b18-82fb-a7273a6300df/kuadrant-console-plugin/0.log" Nov 04 05:46:45.541891 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:45.541854 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5488c58ff5-nc2z6_b5548164-9095-44ae-820c-dbddc724ecbf/kube-auth-proxy/0.log" Nov 04 05:46:46.523551 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:46.523523 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-7rc4s_87f3858d-ee78-4468-8b48-16073a80b645/istio-proxy/0.log" Nov 04 05:46:46.857873 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:46.857842 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-hwbs4_8ca67e76-2ec7-4142-a8c3-0493d3a1cc63/istio-proxy/0.log" Nov 04 05:46:47.179127 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:46:47.179041 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-zfft9_c671ccc5-92a6-4b7d-8a92-7955bd3e15fd/istio-proxy/0.log" Nov 04 05:46:57.132172 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:46:57.132137 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:47:13.274565 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:47:13.274535 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-595689b685-gmspv_57c93381-b98d-4d00-baf7-1b31d0f6c433/manager/1.log" Nov 04 05:47:13.513757 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:47:13.513729 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-595689b685-gmspv_57c93381-b98d-4d00-baf7-1b31d0f6c433/manager/1.log" Nov 04 05:47:27.132067 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:47:27.131984 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:47:50.728856 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:47:50.728822 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-595689b685-gmspv_57c93381-b98d-4d00-baf7-1b31d0f6c433/manager/1.log" Nov 04 05:47:50.974438 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:47:50.974413 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-595689b685-gmspv_57c93381-b98d-4d00-baf7-1b31d0f6c433/manager/1.log" Nov 04 05:47:55.174598 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:47:55.174563 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-69rl9_379bdc95-36d7-47f2-9e17-a96e51da6495/global-pull-secret-syncer/0.log" Nov 04 05:47:55.450438 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:47:55.450357 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-m66r7_915dd778-3692-4f41-83d3-cc5d4e73d615/konnectivity-agent/0.log" Nov 04 05:47:55.473203 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:47:55.473174 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-126.us-east-2.compute.internal_fb1cdcff7ca15ea5212ef7ed52c2b8e3/haproxy/0.log" Nov 04 05:47:57.132476 ip-10-0-128-126 kubenswrapper[5904]: E1104 05:47:57.132440 5904 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-skkvk" containerName="extractor" Nov 04 05:48:02.359670 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:02.359639 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-wm6vh_f31c8bc8-3733-4b18-82fb-a7273a6300df/kuadrant-console-plugin/0.log" Nov 04 05:48:03.764582 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:03.764550 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_215ea51a-5f86-4248-8f04-4974bf428872/alertmanager/0.log" Nov 04 05:48:03.788650 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:03.788624 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_215ea51a-5f86-4248-8f04-4974bf428872/config-reloader/0.log" Nov 04 05:48:03.811725 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:03.811698 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_215ea51a-5f86-4248-8f04-4974bf428872/kube-rbac-proxy-web/0.log" Nov 04 05:48:03.834694 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:03.834670 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_215ea51a-5f86-4248-8f04-4974bf428872/kube-rbac-proxy/0.log" Nov 04 05:48:03.858962 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:03.858939 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_215ea51a-5f86-4248-8f04-4974bf428872/kube-rbac-proxy-metric/0.log" Nov 04 05:48:03.881968 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:03.881945 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_215ea51a-5f86-4248-8f04-4974bf428872/prom-label-proxy/0.log" Nov 04 05:48:03.905676 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:03.905655 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_215ea51a-5f86-4248-8f04-4974bf428872/init-config-reloader/0.log" Nov 04 05:48:03.983720 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:03.983691 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-97647_ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15/kube-state-metrics/0.log" Nov 04 05:48:04.005031 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.005006 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-97647_ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15/kube-rbac-proxy-main/0.log" Nov 04 05:48:04.026558 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.026494 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-97647_ddeb2d26-5ccb-48c2-9859-09fb6dfe5e15/kube-rbac-proxy-self/0.log" Nov 04 05:48:04.054100 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.054077 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-765454fc56-b5f6j_32bea234-f8f1-41fa-ac2f-bea97c729d10/metrics-server/0.log" Nov 04 05:48:04.081341 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.081318 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-4s58k_52797b3b-80b4-443e-aa1d-158430ee739e/monitoring-plugin/0.log" Nov 04 05:48:04.198841 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.198812 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dzs66_7374bdb0-a62f-4b1c-9d14-e1097c851ff0/node-exporter/0.log" Nov 04 05:48:04.225155 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.225133 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dzs66_7374bdb0-a62f-4b1c-9d14-e1097c851ff0/kube-rbac-proxy/0.log" Nov 04 05:48:04.248411 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.248386 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dzs66_7374bdb0-a62f-4b1c-9d14-e1097c851ff0/init-textfile/0.log" Nov 04 05:48:04.364674 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.364641 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-m96wd_137f70c1-c852-4c5f-823f-b8f7ed565c00/kube-rbac-proxy-main/0.log" Nov 04 05:48:04.378821 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.378793 5904 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689"] Nov 04 05:48:04.379196 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379184 5904 cpu_manager.go:414] "containerMap: removing container" podUID="e2b7b8df-514a-43c7-b0a9-920332da4dc6" containerName="istio-proxy" Nov 04 05:48:04.379259 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379198 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2b7b8df-514a-43c7-b0a9-920332da4dc6" containerName="istio-proxy" Nov 04 05:48:04.379259 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379211 5904 cpu_manager.go:414] "containerMap: removing container" podUID="1f4fab8f-362d-4834-bd81-a139841c6bcb" containerName="istio-proxy" Nov 04 05:48:04.379259 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379216 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f4fab8f-362d-4834-bd81-a139841c6bcb" containerName="istio-proxy" Nov 04 05:48:04.379259 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379246 5904 cpu_manager.go:414] "containerMap: removing container" podUID="9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" containerName="istio-proxy" Nov 04 05:48:04.379259 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379251 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" containerName="istio-proxy" Nov 04 05:48:04.379259 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379259 5904 cpu_manager.go:414] "containerMap: removing container" podUID="353f0a70-20f0-4878-b7fa-20221f62ebc4" containerName="istio-proxy" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379264 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="353f0a70-20f0-4878-b7fa-20221f62ebc4" containerName="istio-proxy" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379272 5904 cpu_manager.go:414] "containerMap: removing container" podUID="792505fe-6c68-4def-bd32-a41ca2a1fb34" containerName="istio-proxy" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379277 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="792505fe-6c68-4def-bd32-a41ca2a1fb34" containerName="istio-proxy" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379285 5904 cpu_manager.go:414] "containerMap: removing container" podUID="0d9a7888-648f-441c-8f19-281ec23a306e" containerName="payload-processing" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379289 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d9a7888-648f-441c-8f19-281ec23a306e" containerName="payload-processing" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379299 5904 cpu_manager.go:414] "containerMap: removing container" podUID="c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" containerName="istio-proxy" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379303 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" containerName="istio-proxy" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379322 5904 cpu_manager.go:414] "containerMap: removing container" podUID="0b00d56d-9587-42b9-81a1-1d3cb9787842" containerName="istio-proxy" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379326 5904 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b00d56d-9587-42b9-81a1-1d3cb9787842" containerName="istio-proxy" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379391 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2b7b8df-514a-43c7-b0a9-920332da4dc6" containerName="istio-proxy" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379397 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f4fab8f-362d-4834-bd81-a139841c6bcb" containerName="istio-proxy" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379404 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d600ba3-f1dd-4cf8-a2d2-208bc502aa42" containerName="istio-proxy" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379411 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="353f0a70-20f0-4878-b7fa-20221f62ebc4" containerName="istio-proxy" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379416 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="792505fe-6c68-4def-bd32-a41ca2a1fb34" containerName="istio-proxy" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379422 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d9a7888-648f-441c-8f19-281ec23a306e" containerName="payload-processing" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379429 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="c81ca4c7-8ce0-4591-bf5b-ef6f6ff5a8d1" containerName="istio-proxy" Nov 04 05:48:04.379444 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.379437 5904 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b00d56d-9587-42b9-81a1-1d3cb9787842" containerName="istio-proxy" Nov 04 05:48:04.382683 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.382664 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.391472 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.391450 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z96kl\"/\"kube-root-ca.crt\"" Nov 04 05:48:04.391934 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.391914 5904 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z96kl\"/\"default-dockercfg-8kdm5\"" Nov 04 05:48:04.392032 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.391965 5904 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z96kl\"/\"openshift-service-ca.crt\"" Nov 04 05:48:04.392679 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.392662 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689"] Nov 04 05:48:04.404122 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.404103 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-m96wd_137f70c1-c852-4c5f-823f-b8f7ed565c00/kube-rbac-proxy-self/0.log" Nov 04 05:48:04.427833 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.427811 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-m96wd_137f70c1-c852-4c5f-823f-b8f7ed565c00/openshift-state-metrics/0.log" Nov 04 05:48:04.454978 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.454944 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/929ec87c-d6c1-4255-9cb3-bf38830f7239-lib-modules\") pod \"perf-node-gather-daemonset-bt689\" (UID: \"929ec87c-d6c1-4255-9cb3-bf38830f7239\") " pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.455135 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.455010 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/929ec87c-d6c1-4255-9cb3-bf38830f7239-proc\") pod \"perf-node-gather-daemonset-bt689\" (UID: \"929ec87c-d6c1-4255-9cb3-bf38830f7239\") " pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.455135 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.455090 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/929ec87c-d6c1-4255-9cb3-bf38830f7239-sys\") pod \"perf-node-gather-daemonset-bt689\" (UID: \"929ec87c-d6c1-4255-9cb3-bf38830f7239\") " pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.455135 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.455129 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qbml\" (UniqueName: \"kubernetes.io/projected/929ec87c-d6c1-4255-9cb3-bf38830f7239-kube-api-access-5qbml\") pod \"perf-node-gather-daemonset-bt689\" (UID: \"929ec87c-d6c1-4255-9cb3-bf38830f7239\") " pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.455280 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.455194 5904 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/929ec87c-d6c1-4255-9cb3-bf38830f7239-podres\") pod \"perf-node-gather-daemonset-bt689\" (UID: \"929ec87c-d6c1-4255-9cb3-bf38830f7239\") " pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.460121 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.460102 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3160ae8d-b254-4663-82e5-689c620f553c/prometheus/0.log" Nov 04 05:48:04.482644 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.482616 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3160ae8d-b254-4663-82e5-689c620f553c/config-reloader/0.log" Nov 04 05:48:04.511868 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.511838 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3160ae8d-b254-4663-82e5-689c620f553c/thanos-sidecar/0.log" Nov 04 05:48:04.534441 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.534409 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3160ae8d-b254-4663-82e5-689c620f553c/kube-rbac-proxy-web/0.log" Nov 04 05:48:04.556118 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.556085 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/929ec87c-d6c1-4255-9cb3-bf38830f7239-proc\") pod \"perf-node-gather-daemonset-bt689\" (UID: \"929ec87c-d6c1-4255-9cb3-bf38830f7239\") " pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.556265 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.556143 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/929ec87c-d6c1-4255-9cb3-bf38830f7239-sys\") pod \"perf-node-gather-daemonset-bt689\" (UID: \"929ec87c-d6c1-4255-9cb3-bf38830f7239\") " pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.556265 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.556184 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qbml\" (UniqueName: \"kubernetes.io/projected/929ec87c-d6c1-4255-9cb3-bf38830f7239-kube-api-access-5qbml\") pod \"perf-node-gather-daemonset-bt689\" (UID: \"929ec87c-d6c1-4255-9cb3-bf38830f7239\") " pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.556265 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.556220 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/929ec87c-d6c1-4255-9cb3-bf38830f7239-proc\") pod \"perf-node-gather-daemonset-bt689\" (UID: \"929ec87c-d6c1-4255-9cb3-bf38830f7239\") " pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.556406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.556222 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/929ec87c-d6c1-4255-9cb3-bf38830f7239-podres\") pod \"perf-node-gather-daemonset-bt689\" (UID: \"929ec87c-d6c1-4255-9cb3-bf38830f7239\") " pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.556406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.556269 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/929ec87c-d6c1-4255-9cb3-bf38830f7239-sys\") pod \"perf-node-gather-daemonset-bt689\" (UID: \"929ec87c-d6c1-4255-9cb3-bf38830f7239\") " pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.556406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.556298 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/929ec87c-d6c1-4255-9cb3-bf38830f7239-podres\") pod \"perf-node-gather-daemonset-bt689\" (UID: \"929ec87c-d6c1-4255-9cb3-bf38830f7239\") " pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.556406 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.556358 5904 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/929ec87c-d6c1-4255-9cb3-bf38830f7239-lib-modules\") pod \"perf-node-gather-daemonset-bt689\" (UID: \"929ec87c-d6c1-4255-9cb3-bf38830f7239\") " pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.556593 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.556448 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/929ec87c-d6c1-4255-9cb3-bf38830f7239-lib-modules\") pod \"perf-node-gather-daemonset-bt689\" (UID: \"929ec87c-d6c1-4255-9cb3-bf38830f7239\") " pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.556886 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.556869 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3160ae8d-b254-4663-82e5-689c620f553c/kube-rbac-proxy/0.log" Nov 04 05:48:04.563374 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.563346 5904 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qbml\" (UniqueName: \"kubernetes.io/projected/929ec87c-d6c1-4255-9cb3-bf38830f7239-kube-api-access-5qbml\") pod \"perf-node-gather-daemonset-bt689\" (UID: \"929ec87c-d6c1-4255-9cb3-bf38830f7239\") " pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.580555 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.580536 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3160ae8d-b254-4663-82e5-689c620f553c/kube-rbac-proxy-thanos/0.log" Nov 04 05:48:04.603590 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.603572 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3160ae8d-b254-4663-82e5-689c620f553c/init-config-reloader/0.log" Nov 04 05:48:04.695750 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.695676 5904 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:04.810309 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.810278 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-99867478d-zstlh_e382c8f0-2b69-4271-8098-8328ac9b53ed/thanos-query/0.log" Nov 04 05:48:04.870793 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.870771 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-99867478d-zstlh_e382c8f0-2b69-4271-8098-8328ac9b53ed/kube-rbac-proxy-web/0.log" Nov 04 05:48:04.930681 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.930655 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-99867478d-zstlh_e382c8f0-2b69-4271-8098-8328ac9b53ed/kube-rbac-proxy/0.log" Nov 04 05:48:04.966433 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.966412 5904 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689"] Nov 04 05:48:04.968441 ip-10-0-128-126 kubenswrapper[5904]: W1104 05:48:04.968416 5904 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod929ec87c_d6c1_4255_9cb3_bf38830f7239.slice/crio-77dc242376c168798ba1fc901ac4a081012e3688f23fcb8c1e8e70a222d024cb WatchSource:0}: Error finding container 77dc242376c168798ba1fc901ac4a081012e3688f23fcb8c1e8e70a222d024cb: Status 404 returned error can't find the container with id 77dc242376c168798ba1fc901ac4a081012e3688f23fcb8c1e8e70a222d024cb Nov 04 05:48:04.979146 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:04.979122 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" event={"ID":"929ec87c-d6c1-4255-9cb3-bf38830f7239","Type":"ContainerStarted","Data":"77dc242376c168798ba1fc901ac4a081012e3688f23fcb8c1e8e70a222d024cb"} Nov 04 05:48:05.010589 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:05.010566 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-99867478d-zstlh_e382c8f0-2b69-4271-8098-8328ac9b53ed/prom-label-proxy/0.log" Nov 04 05:48:05.090840 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:05.090814 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-99867478d-zstlh_e382c8f0-2b69-4271-8098-8328ac9b53ed/kube-rbac-proxy-rules/0.log" Nov 04 05:48:05.160720 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:05.160691 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-99867478d-zstlh_e382c8f0-2b69-4271-8098-8328ac9b53ed/kube-rbac-proxy-metrics/0.log" Nov 04 05:48:05.985937 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:05.985900 5904 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" event={"ID":"929ec87c-d6c1-4255-9cb3-bf38830f7239","Type":"ContainerStarted","Data":"bc34869b2645c8c19e9f92f117d84b65b35090562ea45ccddeb48d60f6f11ff6"} Nov 04 05:48:05.986459 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:05.986050 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:06.000905 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:06.000859 5904 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" podStartSLOduration=2.000845825 podStartE2EDuration="2.000845825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:48:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:48:06.000129676 +0000 UTC m=+1132.957714646" watchObservedRunningTime="2026-11-04 05:48:06.000845825 +0000 UTC m=+1132.958430774" Nov 04 05:48:08.885027 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:08.884995 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9zfz8_e19ce810-1d70-46db-a8ce-d824a9ffc81e/dns/0.log" Nov 04 05:48:08.907130 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:08.907091 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9zfz8_e19ce810-1d70-46db-a8ce-d824a9ffc81e/kube-rbac-proxy/0.log" Nov 04 05:48:09.032925 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:09.032896 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fvxf5_80902534-76da-4e9d-8157-ce481e71f2f8/dns-node-resolver/0.log" Nov 04 05:48:09.696905 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:09.696875 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rkg7g_4bcebbdb-3bcd-4050-94b5-3139d6cf3d01/node-ca/0.log" Nov 04 05:48:11.416757 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:11.416733 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5488c58ff5-nc2z6_b5548164-9095-44ae-820c-dbddc724ecbf/kube-auth-proxy/0.log" Nov 04 05:48:11.636706 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:11.636670 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-7rc4s_87f3858d-ee78-4468-8b48-16073a80b645/istio-proxy/0.log" Nov 04 05:48:11.692733 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:11.692656 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-hwbs4_8ca67e76-2ec7-4142-a8c3-0493d3a1cc63/istio-proxy/0.log" Nov 04 05:48:11.751494 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:11.751453 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-zfft9_c671ccc5-92a6-4b7d-8a92-7955bd3e15fd/istio-proxy/0.log" Nov 04 05:48:12.000965 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:12.000897 5904 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-z96kl/perf-node-gather-daemonset-bt689" Nov 04 05:48:12.877481 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:12.877449 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2k9r5_9f075074-0903-4de4-b247-59a26f5b5f95/serve-healthcheck-canary/0.log" Nov 04 05:48:13.568440 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:13.568399 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-skkvk_c93278df-81cb-480e-9151-ba3056f43d74/kube-rbac-proxy/0.log" Nov 04 05:48:13.592080 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:13.592056 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-skkvk_c93278df-81cb-480e-9151-ba3056f43d74/exporter/0.log" Nov 04 05:48:13.614637 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:13.614617 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-skkvk_c93278df-81cb-480e-9151-ba3056f43d74/extractor/0.log" Nov 04 05:48:15.484907 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:15.484881 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5488c58ff5-nc2z6_b5548164-9095-44ae-820c-dbddc724ecbf/kube-auth-proxy/0.log" Nov 04 05:48:15.582589 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:15.582562 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-7rc4s_87f3858d-ee78-4468-8b48-16073a80b645/istio-proxy/0.log" Nov 04 05:48:15.602130 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:15.602106 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-hwbs4_8ca67e76-2ec7-4142-a8c3-0493d3a1cc63/istio-proxy/0.log" Nov 04 05:48:15.635047 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:15.635021 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-zfft9_c671ccc5-92a6-4b7d-8a92-7955bd3e15fd/istio-proxy/0.log" Nov 04 05:48:16.447067 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:16.447037 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-56cd9ffd8-ph9j8_ac9c1355-171c-413a-9e9a-6c6d3dc92da1/manager/0.log" Nov 04 05:48:16.469941 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:16.469919 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-56cd9ffd8-ph9j8_ac9c1355-171c-413a-9e9a-6c6d3dc92da1/copy-manifests/0.log" Nov 04 05:48:17.865873 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:17.865836 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-operator-674cd59dc5-cw8cs_6041142f-186a-4fef-b822-b709112015a0/jobset-operator/0.log" Nov 04 05:48:18.076868 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:18.076840 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5488c58ff5-nc2z6_b5548164-9095-44ae-820c-dbddc724ecbf/kube-auth-proxy/0.log" Nov 04 05:48:18.162032 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:18.161961 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-7rc4s_87f3858d-ee78-4468-8b48-16073a80b645/istio-proxy/0.log" Nov 04 05:48:18.180322 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:18.180295 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-hwbs4_8ca67e76-2ec7-4142-a8c3-0493d3a1cc63/istio-proxy/0.log" Nov 04 05:48:18.215982 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:18.215957 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-zfft9_c671ccc5-92a6-4b7d-8a92-7955bd3e15fd/istio-proxy/0.log" Nov 04 05:48:18.503084 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:18.503010 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-9m4gc_95e8955f-5e49-4bfb-8511-d2a57effca60/manager/0.log" Nov 04 05:48:18.538632 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:18.538601 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-nrwnc_4c46526a-a778-4be1-bdf7-fc3c1f7d38b2/manager/0.log" Nov 04 05:48:18.679825 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:18.679776 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-l8cn6_1f43099e-9c80-4bc5-9a7a-18d3e22534f9/manager/0.log" Nov 04 05:48:18.708898 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:18.708865 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-595689b685-gmspv_57c93381-b98d-4d00-baf7-1b31d0f6c433/manager/0.log" Nov 04 05:48:18.734150 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:18.734120 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-595689b685-gmspv_57c93381-b98d-4d00-baf7-1b31d0f6c433/manager/1.log" Nov 04 05:48:19.110596 ip-10-0-128-126 kubenswrapper[5904]: I1104 05:48:19.110565 5904 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-mkgsm_36073709-5d75-4ba9-9adf-fb35c540843b/proxy/0.log"